Telecommunication Systems

, Volume 71, Issue 4, pp 615–626 | Cite as

DM-CSAT: a LTE-U/Wi-Fi coexistence solution based on reinforcement learning

  • Pedro M. de SantanaEmail author
  • Vicente A. de SousaJr.
  • Fuad M. AbinaderJr.
  • José M. de C. Neto


Recent literature demonstrated promising results of Long-Term Evolution (LTE) deployments over unlicensed bands when coexisting with Wi-Fi networks via the Duty-Cycle (DC) approach. However, it is known that performance in coexistence is strongly dependent on traffic patterns and on the duty-cycle ON–OFF rate of LTE. Most DC solutions rely on static coexistence parameters configuration, hence real-life performance in dynamically varying scenarios might be affected. Advanced reinforcement learning techniques may be used to adjust DC parameters towards efficient coexistence, and we propose a Q-learning Carrier-Sensing Adaptive Transmission mechanism which adapts LTE duty-cycle ON–OFF time ratio to the transmitted data rate, aiming at maximizing the Wi-Fi and LTE-Unlicensed (LTE-U) aggregated throughput. The problem is formulated as a Markov decision process, and the Q-learning solution for finding the best LTE-U ON–OFF time ratio is based on the Bellman’s equation. We evaluate the performance of the proposed solution for different traffic load scenarios using the ns-3 simulator. Results demonstrate the benefits from the adaptability to changing circumstances of the proposed method in terms of Wi-Fi/LTE aggregated throughput, as well as achieving a fair coexistence.


LTE LTE-LAA LTE-U Wi-Fi Q-Learning 



  1. 1.
    GSA, LTE in unlicensed spectrum: Trials, deployments and devices. Technical report, Global Mobile Suppliers Association (2018).Google Scholar
  2. 2.
    Zhang, J., Wang, M., Hua, M., Xia, T., Yang, W., & You, X. (2018). LTE on license-exempt spectrum. IEEE Communications Surveys and Tutorials, 20(1), 647. Scholar
  3. 3.
    Qualcomm. (2014). Qualcomm research LTE in unlicensed spectrum: Harmonious coexistence with Wi-Fi. Technical report, Alcatel-Lucent, Ericsson, Qualcomm Technologies. Accessed 27 Dec 2018.
  4. 4.
    3GPP. (2015). TR 36.889: Feasibility study on licensed-assisted access to unlicensed spectrum (Release 13).Google Scholar
  5. 5.
    Almeida, E., Cavalcante, A. M., Paiva, R. C. D., Chaves, F. S., Abinader, F. M., Vieira, R. D., Choudhury, S., Tuomaala, E., & Doppler, K. (2013). In IEEE ICC 2013.
  6. 6.
    Qualcomm. (2016). Multefire: LTE-like performance with Wi-Fi-like deployment simplicity. Accessed 27 Dec 2018.
  7. 7.
    Alliance, C. (2017). What is CBRS? LTE in 3.5 GHz shared spectrum and what it means for IoT. Accessed 27 Dec 2018.
  8. 8.
    3GPP. (2016). Collaboration on LTE-LWAN integration. Accessed 27 Dec 2018.
  9. 9.
    3GPP. (2016). TR 36.300: LTE; evolved universal terrestrial radio access (e-utra) and evolved universal terrestrial radio access network (e-utran); overall description; stage 2 (Release 13).Google Scholar
  10. 10.
    3GPP. (2017). 3GPP Release 14. Accessed 27 Dec 2018.
  11. 11.
    Sirotkin, S. (2016). LTE-WLAN aggregation (LWA): Benefits and deployment considerations. Accessed 27 Dec 2018.
  12. 12.
    Levy, J. (2016). 802.11 discussions of inputs to 802 EC 5G SC. Accessed 27 Dec 2018.
  13. 13.
    3GPP. (2017). 3GPP work item description: Inclusion of WLAN direct discovery technologies as an alternative for ProSe direct discovery. Accessed 27 Dec 2018.
  14. 14.
    3GPP. (2017). 3GPP work item description: Complementary features for voice services over WLAN. Accessed 27 Dec 2018.
  15. 15.
    Babaei, A., Andreoli-Fang, J., Pang, Y., & Hamzeh, B. (2015). On the impact of LTE-U on Wi-Fi performance. International Journal of Wireless Information Networks, 22(4), 336. Scholar
  16. 16.
    Wang, X., Quek, T. Q. S., Sheng, M., & Li, J. (2016). Throughput and fairness analysis of Wi-Fi and LTE-U in unlicensed band. IEEE Journal on Selected Areas in Communications.
  17. 17.
    Cano, C., & Leith, D. J. (2015). Coexistence of Wi-Fi and LTE in unlicensed bands: A proportional fair allocation scheme.
  18. 18.
    Pang, Y., Babaei, A., Andreoli-Fang, J., & Hamzeh, B. (2017). Wi-Fi coexistence with duty cycled LTE-U. Wireless Communications and Mobile Computing, 2017, 1. Scholar
  19. 19.
    Wi-Fi Alliance. (2016). Draft coexistence test plan—v0.8.4—alpha. Technical report. Accessed 27 Dec 2018.
  20. 20.
    Pang, Y., Babaei, A., Andreoli-Fang, J., & Hamzeh, B. (2017). Wi-Fi coexistence with duty cycled LTE-U. Wireless Communications and Mobile Computing, 2017, 6486380.Google Scholar
  21. 21.
    Abdelfattah, A., & Malouch, N. (2016). Studying the impact of LTE-U on Wi-Fi downlink performance. In IEEE 12th international conference on wireless and mobile computing, networking and communications (WiMob).Google Scholar
  22. 22.
    Andreoli-Fang, J. (2015). Wi-Fi versus duty cycled LTE-U: in-home testing reveals coexistence challenges. Technical report, Cable Labs. Accessed 27 Dec 2018.
  23. 23.
    C.T. de Telecomunicacions de Catalunya. (2017). SpiderCloud and CTTC model LTE-U performance advancements (CTTC Newsroom). Accessed 27 Dec 2018.
  24. 24.
    Gao, Y., Chu, X., & Zhang, J. (2016). Performance analysis of LAA and WiFi coexistence in unlicensed spectrum based on markov chain. In IEEE GLOBECOM.
  25. 25.
    Cano, C., Leith, D. J., Garcia-Saavedra, A., & Serrano, P. (2016). Fair coexistence of scheduled and random access wireless networks: Unlicensed LTE/WiFi. IEEE/ACM Transactions on Networking.
  26. 26.
    Chen, B., Chen, J., Gao, Y., & Zhang, J. (2017). Coexistence of LTE-LAA and Wi-Fi on 5 GHz with corresponding deployment scenarios: A survey. IEEE Communications Surveys and Tutorials.
  27. 27.
    El-Samadisy, O., Khedr, M., & El-Helw, A. (2016). Performance evaluation of MAC for IEEE 802.11 and LAA LTE. In International conference on computational science and computational intelligence.
  28. 28.
    Dama, S., Kumar, A., & Kuchi, K. (2015). Performance evaluation of LAA-LBT based LTE and WLANs co-existence in unlicensed spectrum. In Globecom workshops.
  29. 29.
    Rupasinghe, N., & Guvenc, I. (2014). Licensed-assisted access for WiFi-LTE coexistence in the unlicensed spectrum. In Globecom workshops emerging technologies for 5G wireless cellular networks. ISBN 978-1-4799-7470-2/14.Google Scholar
  30. 30.
    Kwan, R., Pazhyannur, R., & Chandrasekhar, V. (2015). Fair co-existence of Licensed Assisted Access LTE (LAA-LTE) and Wi-Fi in unlicensed spectrum. In 7th CEEC. ISBN 978-1-4673-9481-9.Google Scholar
  31. 31.
    de Santana, P. M., Melo, V. D. D. L., & De Sousa, V. A., Jr. (2016). Performance of License Assisted Access solutions using ns-3. In CSCI.
  32. 32.
    Maglogiannis, V., Naudts, D., Shahid, A., & Moerman, I. (2018). An adaptive LTE listen-before-talk scheme towards a fair coexistence with Wi-Fi in unlicensed spectrum. Telecommunication Systems.
  33. 33.
    Galanopoulos, A., Foukalas, F., & Tsiftsis, T. A. (2016). Efficient coexistence of LTE with WiFi in the licensed and unlicensed spectrum aggregation. IEEE Transactions on Cognitive Communication and Networking.
  34. 34.
    Castane, A., Perez-Romero, & Sallent, O. (2017). On the implementation of channel selection for LTE in unlicensed bands using Q-learning and game theory algorithms. In Wireless communications and mobile computing conference.
  35. 35.
    Chen, M., Saad, W., & Yin, C. (2016). Echo state networks for self-organizing resource allocation in LTE-u with uplink-downlink decoupling. IEEE Transactions on Wireless Communications., 16, 3–16.CrossRefGoogle Scholar
  36. 36.
    Rupasinghe, N., & Guvenc, I. (2015). Reinforcement learning for Licensed-Assisted Access of LTE in the unlicensed spectrum. In Wireless communications and networking conference.Google Scholar
  37. 37.
    Feinberg, E. A., & Shwartz, A. (2002). Handbook of Markov decision processes (1st ed.). New York: Springer.CrossRefGoogle Scholar
  38. 38.
    Stroock, D. W. (2014). An introduction to Markov processes (2nd ed.). Berlin: Springer.CrossRefGoogle Scholar
  39. 39.
    Haykin, S. O. (2008). Neural networks and learning machines (3rd ed.). London: Pearson.Google Scholar
  40. 40.
    Bellman, R. E. (2010). Dynamic programming (1st ed.). Princeton: Princeton University Press.Google Scholar
  41. 41.
    Watkins, C. (1989). Learning from delayed rewards. Ph.D. thesis, University of Cambridge, London.Google Scholar
  42. 42.
    Abinader, F. M., Almeida, E. P. L., Chaves, F. S., Cavalcante, A. M., Vieira, R. D., Paiva, R. C. D., et al. (2014). Enabling the coexistence of LTE and Wi-Fi in unlicensed bands. IEEE Communications Magazine, 52(11), 54. Scholar
  43. 43.
    3GPP. (2015). Tr 36.814: Technical specification group radio access network; evolved universal terrestrial radio access (e-utra); further advancements for e-utra physical layer aspects (release 9).Google Scholar
  44. 44.
    Koenig, S., & Simmons, R. G. (1993). Complexity analysis of real-time reinforcement learning. In Proceedings of the 11th national conference on artificial intelligence.Google Scholar
  45. 45.
    Watkins, C. J., & Dayan, P. (1992). Technical note: Q-learning. Journal of Machine Learning, 8(3–4), 279.Google Scholar
  46. 46.
    ns-3. (2016). ns-3 website. Accessed 27 Dec 2018.
  47. 47.
    ns-3. (2016). ns-3 repositories. Accessed 27 Dec 2018.
  48. 48.
    ns-3. (2016). License-Assisted Access - ns-3 project. Accessed 27 Dec 2018.
  49. 49.
    ns-3. (2018). ns-3 app store. Accessed 27 Dec 2018.
  50. 50.
    3GPP. (2011). TS 36.423: LTE E-UTRAN X2 application protocol (X2AP), (Release 10).Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Samsung R&D Institute (SIDIA)ManausBrazil
  2. 2.Federal University of Rio Grande do Norte (UFRN)NatalBrazil

Personalised recommendations