Abstract
Wireless local area networks (WLANs) are widely deployed for Internet-centric data applications. Due to their extensive norm in our day-to-day wireless-enabled life, WLANs are expected to play a vital role for Internet of Things (IoT). It is predicted that by 2020, about 50 billion devices (things) will be connected via IoT. Consequently, WLANs need major improvements in both throughput and efficiency for such a massive device deployment in applications like smart offices, smart train stations, and smart stadiums for smart cities in IoT. New technologies continue to be introduced for WLAN applications for this purpose. The IEEE 802.11ac standard is the currently implemented amendment by the IEEE 802.11 standard working group that promises data rates at gigabits per second. The main features of the IEEE 802.11ac standard are adopting increased bandwidth and higher order modulation than the previous standards, and multiple-input, multiple-output (MIMO) and multiuser MIMO transmission modes. These features are designed to improve the user experience. In addition to technologies that enhance the efficiency of the WLAN, the IEEE 802.11ax High Efficiency WLAN (HEW) standard is also investigating and evaluating advanced wireless technologies to utilize the existing spectrum more efficiently.
The next-generation dense WLAN, HEW is expected to confront ultradense user environments and radically new applications for smart cities. HEW is likely to provide four times higher network efficiency even in highly dense network deployments. However, the current WLAN itself faces huge challenge of efficient channel access due to its temporal-based MAC layer resource allocation (MAC-RA). WLAN uses a carrier sense multiple access with collision avoidance (CSMA/CA) procedure to access the channel resources, which is based on a binary exponential backoff (BEB) mechanism. In BEB, a random backoff value is generated from a contention window (CW) to obtain channel access. The CW size is doubled after every unsuccessful transmission and reset to its minimum value on successfully transmissions. However, this blindness when increasing and resetting the CW induces performance degradation. For a dense network, resetting the CW to its minimum size may result in more collisions and poor network performance. Likewise, for a small network environment, a blind increase in CW size may cause an unnecessarily long delay while accessing the channel. To satisfy the diverse requirements of dense WLANs, it is anticipated that prospective HEW will autonomously access the best channel resources with the assistance of sophisticated wireless channel condition inference in order to control channel collisions. Such intelligence is possible with the introduction of deep learning (DL) techniques in future WLANs.
The potential applications of DL to the MAC layer of IEEE 802.11 standards have already been progressively acknowledged due to their novel features for future communications. Their new features challenge conventional communications theories with more sophisticated artificial intelligence-based theories. DL has been extensively proposed for the MAC layer of WLANs in various research areas, such as deployment of cognitive radio and communications networks. Deep reinforcement learning (DRL) is one of the DL techniques that are motivated by the behaviorist sensibility and control philosophy, where a learner can achieve an objective by interacting with the environment. In this chapter, a DRL-based intelligent paradigm is developed for channel access in dense WLANs in smart cities.
One of the DRL models, Q-learning (QL), is used to propose an intelligent QL-based resource allocation (iQRA) mechanism for MAC-RA in dense wireless networks. iQRA exploits channel observation-based collision probability for network inference to dynamically and autonomously control the backoff parameters (such as backoff stages and CW values). The simulations performed in network simulator 3 (ns3) indicate that the proposed DRL-based iQRA paradigm learns diverse WLAN environments and optimizes its performance, compared to conventional non-intelligent MAC protocol, BEB. The performance of the proposed iQRA mechanism is evaluated in diverse WLAN network environments with throughput, channel access delay, and fairness as performance metrics.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
IEEE 802.11ax High Efficiency WLAN (HEW), P802.11-TGax, 2014.
Ali, R., Kim, S. W., Kim, B., & Park, Y. (2018, February). Design of MAC layer resource allocation schemes for IEEE 802.11ax: Future directions. IETE Technical Review, 35(1), 28–52. https://doi.org/10.1080/02564602.2016.1242387
Ali, R., Shahin, N., Bajracharya, R., Kim, Y. T., Kim, B., & Kim, S. W. (2018). A self-scrutinized backoff mechanism for IEEE 802.11ax in 5G unlicensed networks. Sustainability, 10(4), 1201. https://doi.org/10.3390/su10041201
Moon, J., & Lim Y (2017). A reinforcement learning approach to access management in wireless cellular networks. Wireless Communications and Mobile Computing, 2017, Article ID 6474768, 7 pages. https://doi.org/10.1155/2017/6474768
Jiang, C., Zhang, H., Ren, Y., Han, Z., Chen, K., & Hanzo, L. (2017, April). Machine learning paradigms for next-generation wireless networks. IEEE Wireless Communications, 24(2), 98–105. https://doi.org/10.1109/MWC.2016.1500356WC
Primer. (2013). Wi-Fi: overview of the 802.11 physical layer and transmitter measurements (pp. 4–7). Beaverton: Tektronix Inc.
Gong, M. X., Hart, B., & Mao, S. (2015, January). Advanced wireless LAN Technologies: IEEE 802.11ac and beyond. GetMobile: Mobile Computing and Communications, 18(4), 48–52.
Charfi, E., Chaariand, L., & Kamoun, L. (2013, November). PHY/MAC enhancements and QoS mechanisms for very high throughput WLANs: A survey. IEEE Communications Surveys and Tutorials, 15(4), 1714–1735.
Barber, P. (2013). IEEE 802.11ax project plan, IEEE technical presentation [Online]. Available: http://www.ieee802.org/11/Reports/tgax_update.htm
Yunoki, K., & Misawa, Y. (2013). Possible approaches for HEW, IEEE technical presentation [Online]. Available: http://www.ieee802.org/11/Reports/tgax_update.htm
IEEE MAC Enhancement for Quality of Service. IEEE Standard 802.11e, 2005.
Yu, X., Navaratnam, P., & Moessner, K. (2013, July). Resource reservation schemes for IEEE 802.11-based wireless networks: A survey. IEEE Communications Surveys and Tutorials, 15(3), 1042–1061.
Youssef, M. A., & Miller, R. E. (2002). Analyzing the point coordination function of the IEEE 802.11 WLAN protocol using a systems of communicating machines specification, UMIACS Technical Report CS-TR-4357 (p. 36). College Park, MD: UM Computer Science Dept.
IEEE Standard for Information Technology. Part 11: Wireless LAN medium access control (MAC) and physical layer (PHY) specifications. ANSI/IEEE Std 802.11 2007, i-513.
Li, R., Zhao, Z., Zhou, X., Ding, G., Chen, Y., Wang, Z., et al. (2017). Intelligent 5G: When cellular networks meet artificial intelligence. IEEE Wireless Communications, 24(5), 175–183. https://doi.org/10.1109/MWC.2017.1600304WC
Alpaydin, E. (2014). Introduction to machine learning (3rd ed.). Cambridge, MA: MIT Press. ISBN: 978-0-262-028189.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (2nd ed.). Cambridge, MA: MIT Press. ISBN: 0262193981.
Aprem, A., Murthy, C. R., & Mehta, N. B. (2013). Transmit power control policies for energy harvesting sensors with retransmissions. IEEE Journal of Selected Topics in Signal Processing, 7(5), 895–906. https://doi.org/10.1109/JSTSP.2013.2258656
Alnwaimi, G., Vahid, S., & Moessner, K. (2015). Dynamic heterogeneous learning games for opportunistic access in LTE-based macro/femtocell deployments. IEEE Transactions on Wireless Communications, 14(4), 2294–2308. https://doi.org/10.1109/TWC.2014.2384510
Ali, R., Shahin, N., Kim, Y. T., Kim, B., & Kim, S. W. (2018, May). Channel observation-based scaled backoff mechanism for high efficiency WLANs. Electronics Letters, 54(10), 663–665.
The Network Simulator — ns-3 [Online]. Available: https://www.nsnam.org/
Jain, R., Chiu, D., & Hawe, W. (1984). A quantitative measure of fairness and discrimination for resource allocation in shared computer system. Hudson: Eastern Research Laboratory, Digital Equipment Corporation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix: SCI/SCIE Journal Publications Related to the Chapter
Appendix: SCI/SCIE Journal Publications Related to the Chapter
-
1.
R. Ali, S. W. Kim, B. Kim, and Y. Park, “Design of MAC layer resource allocation schemes for IEEE 802.11ax: Future directions,” IETE Technical Review, vol. 35(1), pp. 28–52, February 2018.
-
2.
R. Ali, N. Shahin, R. Bajracharya, B. Kim and S. W. Kim, “A self-scrutinized backoff mechanism for IEEE 802.11ax in 5G unlicensed networks,” Sustainability, vol. 10(4), pp. 1–15, April 2018.
-
3.
R. Ali, N. Shahin, Y. Kim, B. Kim and S. W. Kim, “Channel observation-based scaled backoff mechanism for high-efficiency WLANs,” Electronics Letters, vol. 54(10), pp. 663–665, May 2018.
-
4.
R. Ali, N. Shahin, Y. B. Zikria, B. Kim, and S. W. Kim, “Deep reinforcement learning paradigm for performance optimization of channel observation-based MAC protocols in dense WLANs,” IEEE Access, (accepted).
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Ali, R., Zikria, Y.B., Kim, BS., Kim, S.W. (2020). Deep Reinforcement Learning Paradigm for Dense Wireless Networks in Smart Cities. In: Al-Turjman, F. (eds) Smart Cities Performability, Cognition, & Security. EAI/Springer Innovations in Communication and Computing. Springer, Cham. https://doi.org/10.1007/978-3-030-14718-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-14718-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-14717-4
Online ISBN: 978-3-030-14718-1
eBook Packages: EngineeringEngineering (R0)