Skip to main content
SpringerLink
Log in

DM-CSAT: a LTE-U/Wi-Fi coexistence solution based on reinforcement learning

  • Published:
Telecommunication Systems Aims and scope Submit manuscript

Abstract

Recent literature demonstrated promising results of Long-Term Evolution (LTE) deployments over unlicensed bands when coexisting with Wi-Fi networks via the Duty-Cycle (DC) approach. However, it is known that performance in coexistence is strongly dependent on traffic patterns and on the duty-cycle ON–OFF rate of LTE. Most DC solutions rely on static coexistence parameters configuration, hence real-life performance in dynamically varying scenarios might be affected. Advanced reinforcement learning techniques may be used to adjust DC parameters towards efficient coexistence, and we propose a Q-learning Carrier-Sensing Adaptive Transmission mechanism which adapts LTE duty-cycle ON–OFF time ratio to the transmitted data rate, aiming at maximizing the Wi-Fi and LTE-Unlicensed (LTE-U) aggregated throughput. The problem is formulated as a Markov decision process, and the Q-learning solution for finding the best LTE-U ON–OFF time ratio is based on the Bellman’s equation. We evaluate the performance of the proposed solution for different traffic load scenarios using the ns-3 simulator. Results demonstrate the benefits from the adaptability to changing circumstances of the proposed method in terms of Wi-Fi/LTE aggregated throughput, as well as achieving a fair coexistence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. We argue that the way we modeled our problem and configured our Q-Learning algorithm does not require a complex reinforcement learning solution. This is two fold. First, the Wi-Fi AP operation mode has a passive influence that is accounted for the MDP just as part of the environment. In other words, it is not a multi-agent competition scenario. In fact, the LTE-U DM-CSAT is the only one to perform decisions. Second, we applied a restricted quantity of states and actions, such that updating the Q-Table does not become ineffective.

References

  1. GSA, LTE in unlicensed spectrum: Trials, deployments and devices. Technical report, Global Mobile Suppliers Association (2018).

  2. Zhang, J., Wang, M., Hua, M., Xia, T., Yang, W., & You, X. (2018). LTE on license-exempt spectrum. IEEE Communications Surveys and Tutorials, 20(1), 647. https://doi.org/10.1109/comst.2017.2771485.

    Article  Google Scholar 

  3. Qualcomm. (2014). Qualcomm research LTE in unlicensed spectrum: Harmonious coexistence with Wi-Fi. Technical report, Alcatel-Lucent, Ericsson, Qualcomm Technologies. https://HrBwww.qualcomm.com/documents/lte-unlicensed-coexistence-whiHrBtepaper. Accessed 27 Dec 2018.

  4. 3GPP. (2015). TR 36.889: Feasibility study on licensed-assisted access to unlicensed spectrum (Release 13).

  5. Almeida, E., Cavalcante, A. M., Paiva, R. C. D., Chaves, F. S., Abinader, F. M., Vieira, R. D., Choudhury, S., Tuomaala, E., & Doppler, K. (2013). In IEEE ICC 2013. https://doi.org/10.1109/ICC.2013.6655388.

  6. Qualcomm. (2016). Multefire: LTE-like performance with Wi-Fi-like deployment simplicity. https://www.qualcomm.com/invention/technologies/lte/multefire. Accessed 27 Dec 2018.

  7. Alliance, C. (2017). What is CBRS? LTE in 3.5 GHz shared spectrum and what it means for IoT. https://www.leverege.com/blogpost/what-is-cbrs-lte-3-5-ghz. Accessed 27 Dec 2018.

  8. 3GPP. (2016). Collaboration on LTE-LWAN integration. http://www.3gpp.org/news-events/3gpp-news/1771-wlan_lte. Accessed 27 Dec 2018.

  9. 3GPP. (2016). TR 36.300: LTE; evolved universal terrestrial radio access (e-utra) and evolved universal terrestrial radio access network (e-utran); overall description; stage 2 (Release 13).

  10. 3GPP. (2017). 3GPP Release 14. http://www.3gpp.org/release-14. Accessed 27 Dec 2018.

  11. Sirotkin, S. (2016). LTE-WLAN aggregation (LWA): Benefits and deployment considerations. https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/lte-wlan-aggregation-deployment-paper.pdf. Accessed 27 Dec 2018.

  12. Levy, J. (2016). 802.11 discussions of inputs to 802 EC 5G SC. https://mentor.ieee.org/802.11/dcn/16/11-16-0651-01-0000-802-11-discussion-of-inputs-to-802-ec-5g-sc.pptx. Accessed 27 Dec 2018.

  13. 3GPP. (2017). 3GPP work item description: Inclusion of WLAN direct discovery technologies as an alternative for ProSe direct discovery. http://www.3gpp.org/ftp/tsg_ct/TSG_CT/TSGC_76_West_Palm_Beach/Docs/CP-171102.zip. Accessed 27 Dec 2018.

  14. 3GPP. (2017). 3GPP work item description: Complementary features for voice services over WLAN. http://www.3gpp.org/ftp/tsg_ct/TSG_CT/TSGC_76_West_Palm_Beach/Docs/CP-171101.zip. Accessed 27 Dec 2018.

  15. Babaei, A., Andreoli-Fang, J., Pang, Y., & Hamzeh, B. (2015). On the impact of LTE-U on Wi-Fi performance. International Journal of Wireless Information Networks, 22(4), 336. https://doi.org/10.1007/s10776-015-0288-6.

    Article  Google Scholar 

  16. Wang, X., Quek, T. Q. S., Sheng, M., & Li, J. (2016). Throughput and fairness analysis of Wi-Fi and LTE-U in unlicensed band. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/jsac.2016.2632629.

  17. Cano, C., & Leith, D. J. (2015). Coexistence of Wi-Fi and LTE in unlicensed bands: A proportional fair allocation scheme. https://doi.org/10.1109/iccw.2015.7247522.

  18. Pang, Y., Babaei, A., Andreoli-Fang, J., & Hamzeh, B. (2017). Wi-Fi coexistence with duty cycled LTE-U. Wireless Communications and Mobile Computing, 2017, 1. https://doi.org/10.1155/2017/6486380.

    Article  Google Scholar 

  19. Wi-Fi Alliance. (2016). Draft coexistence test plan—v0.8.4—alpha. Technical report. https://www.wi-fi.org/file/draft-coexistence-test-plan. Accessed 27 Dec 2018.

  20. Pang, Y., Babaei, A., Andreoli-Fang, J., & Hamzeh, B. (2017). Wi-Fi coexistence with duty cycled LTE-U. Wireless Communications and Mobile Computing, 2017, 6486380.

  21. Abdelfattah, A., & Malouch, N. (2016). Studying the impact of LTE-U on Wi-Fi downlink performance. In IEEE 12th international conference on wireless and mobile computing, networking and communications (WiMob).

  22. Andreoli-Fang, J. (2015). Wi-Fi versus duty cycled LTE-U: in-home testing reveals coexistence challenges. Technical report, Cable Labs. https://www.cablelabs.com/vran-over-docsis-cablelabs-making-reality. Accessed 27 Dec 2018.

  23. C.T. de Telecomunicacions de Catalunya. (2017). SpiderCloud and CTTC model LTE-U performance advancements (CTTC Newsroom). http://www.cttc.es/spidercloud-and-cttc-model-lte-u-performance-advancements/. Accessed 27 Dec 2018.

  24. Gao, Y., Chu, X., & Zhang, J. (2016). Performance analysis of LAA and WiFi coexistence in unlicensed spectrum based on markov chain. In IEEE GLOBECOM. https://doi.org/10.1109/GLOCOM.2016.7842129.

  25. Cano, C., Leith, D. J., Garcia-Saavedra, A., & Serrano, P. (2016). Fair coexistence of scheduled and random access wireless networks: Unlicensed LTE/WiFi. IEEE/ACM Transactions on Networking. https://doi.org/10.1109/TNET.2017.2731377.

  26. Chen, B., Chen, J., Gao, Y., & Zhang, J. (2017). Coexistence of LTE-LAA and Wi-Fi on 5 GHz with corresponding deployment scenarios: A survey. IEEE Communications Surveys and Tutorials. https://doi.org/10.1109/COMST.2016.2593666.

  27. El-Samadisy, O., Khedr, M., & El-Helw, A. (2016). Performance evaluation of MAC for IEEE 802.11 and LAA LTE. In International conference on computational science and computational intelligence. https://doi.org/10.1109/CSCI.2016.177.

  28. Dama, S., Kumar, A., & Kuchi, K. (2015). Performance evaluation of LAA-LBT based LTE and WLANs co-existence in unlicensed spectrum. In Globecom workshops. https://doi.org/10.1109/GLOCOMW.2015.7414071.

  29. Rupasinghe, N., & Guvenc, I. (2014). Licensed-assisted access for WiFi-LTE coexistence in the unlicensed spectrum. In Globecom workshops emerging technologies for 5G wireless cellular networks. ISBN 978-1-4799-7470-2/14.

  30. Kwan, R., Pazhyannur, R., & Chandrasekhar, V. (2015). Fair co-existence of Licensed Assisted Access LTE (LAA-LTE) and Wi-Fi in unlicensed spectrum. In 7th CEEC. ISBN 978-1-4673-9481-9.

  31. de Santana, P. M., Melo, V. D. D. L., & De Sousa, V. A., Jr. (2016). Performance of License Assisted Access solutions using ns-3. In CSCI. https://doi.org/10.1109/CSCI.2016.0181.

  32. Maglogiannis, V., Naudts, D., Shahid, A., & Moerman, I. (2018). An adaptive LTE listen-before-talk scheme towards a fair coexistence with Wi-Fi in unlicensed spectrum. Telecommunication Systems. https://doi.org/10.1007/s11235-017-0418-9.

  33. Galanopoulos, A., Foukalas, F., & Tsiftsis, T. A. (2016). Efficient coexistence of LTE with WiFi in the licensed and unlicensed spectrum aggregation. IEEE Transactions on Cognitive Communication and Networking. https://doi.org/10.1109/TCCN.2016.2594780.

  34. Castane, A., Perez-Romero, & Sallent, O. (2017). On the implementation of channel selection for LTE in unlicensed bands using Q-learning and game theory algorithms. In Wireless communications and mobile computing conference. https://doi.org/10.1109/IWCMC.2017.7986438.

  35. Chen, M., Saad, W., & Yin, C. (2016). Echo state networks for self-organizing resource allocation in LTE-u with uplink-downlink decoupling. IEEE Transactions on Wireless Communications., 16, 3–16.

    Article  Google Scholar 

  36. Rupasinghe, N., & Guvenc, I. (2015). Reinforcement learning for Licensed-Assisted Access of LTE in the unlicensed spectrum. In Wireless communications and networking conference.

  37. Feinberg, E. A., & Shwartz, A. (2002). Handbook of Markov decision processes (1st ed.). New York: Springer.

    Book  Google Scholar 

  38. Stroock, D. W. (2014). An introduction to Markov processes (2nd ed.). Berlin: Springer.

    Book  Google Scholar 

  39. Haykin, S. O. (2008). Neural networks and learning machines (3rd ed.). London: Pearson.

    Google Scholar 

  40. Bellman, R. E. (2010). Dynamic programming (1st ed.). Princeton: Princeton University Press.

    Google Scholar 

  41. Watkins, C. (1989). Learning from delayed rewards. Ph.D. thesis, University of Cambridge, London.

  42. Abinader, F. M., Almeida, E. P. L., Chaves, F. S., Cavalcante, A. M., Vieira, R. D., Paiva, R. C. D., et al. (2014). Enabling the coexistence of LTE and Wi-Fi in unlicensed bands. IEEE Communications Magazine, 52(11), 54. https://doi.org/10.1109/MCOM.2014.6957143.

    Article  Google Scholar 

  43. 3GPP. (2015). Tr 36.814: Technical specification group radio access network; evolved universal terrestrial radio access (e-utra); further advancements for e-utra physical layer aspects (release 9).

  44. Koenig, S., & Simmons, R. G. (1993). Complexity analysis of real-time reinforcement learning. In Proceedings of the 11th national conference on artificial intelligence.

  45. Watkins, C. J., & Dayan, P. (1992). Technical note: Q-learning. Journal of Machine Learning, 8(3–4), 279.

    Google Scholar 

  46. ns-3. (2016). ns-3 website. https://www.nsnam.org/. Accessed 27 Dec 2018.

  47. ns-3. (2016). ns-3 repositories. http://code.nsnam.org/. Accessed 27 Dec 2018.

  48. ns-3. (2016). License-Assisted Access - ns-3 project. https://www.nsnam.org/wiki/LAA-WiFi-Coexistence. Accessed 27 Dec 2018.

  49. ns-3. (2018). ns-3 app store. http://ns-apps.ee.washington.edu/. Accessed 27 Dec 2018.

  50. 3GPP. (2011). TS 36.423: LTE E-UTRAN X2 application protocol (X2AP), (Release 10).

Download references

Author information

Authors and Affiliations

  1. Samsung R&D Institute (SIDIA), Manaus, AM, Brazil

    Pedro M. de Santana & Fuad M. Abinader Jr.

  2. Federal University of Rio Grande do Norte (UFRN), Natal, RN, Brazil

    Vicente A. de Sousa Jr. & José M. de C. Neto

Authors
  1. Pedro M. de Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vicente A. de Sousa Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fuad M. Abinader Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José M. de C. Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro M. de Santana.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The proof of concept simulations provided by this paper was supported by High Performance Computing Center at UFRN (NPAD/UFRN). Fuad M. Abinader Jr. is currently with Nokia Bell Labs, Paris, France. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brasil (CAPES)—Finance Code 001.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Santana, P.M., de Sousa, V.A., Abinader, F.M. et al. DM-CSAT: a LTE-U/Wi-Fi coexistence solution based on reinforcement learning. Telecommun Syst 71, 615–626 (2019). https://doi.org/10.1007/s11235-018-00535-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11235-018-00535-7

Keywords

Search

Navigation