Skip to main content

Advertisement

SpringerLink
Log in

WiGig access point selection using non-contextual and contextual multi-armed bandit in indoor environment

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Millimeter wave (mmWave) band, i.e., 30 ~ 300 GHz, supports multi-gigabit communication making it a main component of fifth generation (5G) and future six generation (6G) wireless communications. Wireless gigabit (WiGig) is the standardized 60 GHz mmWave band for WLAN applications. MmWave has intermittent short-range transmissions necessitating the installation of multiple WiGig access points (APs) using antenna beamforming training (BT) to fully cover a target indoor area. WiGig user equipment (UE) should select the best AP among the installed ones maximizing its achievable data rate. Conventionally, UE should exhaustively search the best AP having the highest received power using BT with all available APs, which reduces the attainable throughput in consequence. In this paper, the problem of WiGig AP selection is formulated as a multi-armed bandit (MAB) game, where, the UE is considered as the player aiming to maximize its long-term average throughput, i.e., the reward, through playing over the available APs, i.e., the arms of the bandit. Non-contextual MAB algorithms, namely upper confidence bound (UCB) and Thompson sampling (TS) are adopted to address the formulated problem. Moreover, as standardized WiGig devices are multi-band capable containing 2.4/5 GHz Wi-Fi and 60 GHz mmWave bands, Wi-Fi signal characteristics are used as contexts of the mmWave links to further enhance the WiGig AP selection policy. Thus, contextual MAB (CMAB) algorithms, namely linear UCB (LinUCB) and contextual TS (CTS) are also suggested. Simulation analyses confirm the superior performance of the CMAB algorithms over the non-contextual ones in addition to the conventional approaches accompanied with high convergence rates. For example, at no blockage and using too narrow beams of \({\theta }_{-3{\text{dB}}}={10}^{^\circ }\), the proposed CTS, LinUCB, TS, and UCB schemes obtain 98.7%, 96.8%, 89%, 84% of the optimal performance, while the benchmark schemes obtain 49%, and 1.8%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Abdelreheem A, Mohamed EM, Esmaiel H (2018) Location-based millimeter wave multi-level beamforming using compressive sensing. IEEE Commun Lett 22(1):185–188

    Article  Google Scholar 

  • Agrawal S, Goyal N (2013) Thompson sampling for contextual bandits with linear payoffs. In: the 30th International Conference on Machine Learning, ICML: 1–9

  • Alkhateeb A, El Ayach O, Leus G, Heath RW Jr (2014) Channel estimation and hybrid precoding for millimeter wave cellular systems. IEEE J Sel Top Signal Process 8(5):831–846

    Article  Google Scholar 

  • Audibert JY, Munos R, Szepesvari C (2009) Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theor Comput Sci 410(19):1876–1902

    Article  MathSciNet  MATH  Google Scholar 

  • Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2):235–256

    Article  MATH  Google Scholar 

  • Chen X, Yuan W, Cheng W, Liu W, Leung H (2013) Access point selection under QoS requirements in variable channel-width WLANs. IEEE Wireless Commun Lett 2(1):114–117

    Article  Google Scholar 

  • Dwijaksara MH, Jeon WS, Jeong DG (2019) User association for load balancing and energy saving in enterprise WLANs. IEEE Syst J 13(3):2700–2711

    Article  Google Scholar 

  • Francisco VI, Marcial-Romero J, Valdovinos R (2019) A comparison between UCB and UCB-tuned as selection policies GGP. IFS 36(5):5073–5079

    Article  Google Scholar 

  • Gao X, Zhang J, Liu G et al (2007) Large-scale characteristics of 5.25 GHz based on wideband MIMO channel measurements. IEEE Antenn Wirel Propag Lett 6:263–266

    Article  Google Scholar 

  • Ghasempour Y, da Silva CR, Cordeiro C, Knightly EW (2017) IEEE 802.11 ay: next-generation 60 GHz communication for 100 Gb/s Wi-Fi. IEEE Commun Magaz 55(12):186–192

    Article  Google Scholar 

  • Gutowski N, Amghar T, Camp O, Chhel F (2018) Context enhancement for linear contextual multi-armed bandits. In: 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI)

  • Hashima S, ElHalawany BM, Hatano K, Wu K, Mohamed EM (2021) Leveraging machine-learning for D2D communications in 5G/beyond 5G networks. Electronics 10(2):1–16

    Article  Google Scholar 

  • Hosoya K, Prasad N, Ramachandran K et al (2015) Multiple sector ID capture (MIDC): a novel beamforming technique for 60 GHz band multi-Gbps WLAN/PAN systems. IEEE Trans Antennas Propag 63(1):81–96

    Article  MathSciNet  MATH  Google Scholar 

  • IEEE 802.11ad Standard (2012) Enhancements for very high throughput in the 60 GHz band

  • Kaufmann E, Korda N, Munos R (2012) Thompson sampling: An asymptotically optimal finite-time analysis. In: International Conference on Algorithmic Learning Theory: 199–213

  • Khishea M, Mosavib MR (2020) Chimp optimization algorithm. Expert Syst Appl 149(11338):1–26

    Google Scholar 

  • Kim H, Lee W, Bae M, Kim H (2017) Wi-Fi seeker: a link and load aware AP selection algorithm. IEEE Trans Mob Comput 16(8):2366–2378

    Article  Google Scholar 

  • Lihong L, Wei C, Langford J, Schapire R (2010) A contextual-bandit approach to personalized news article recommendation. In: the19th international conference on World wide web: 661–670

  • Liu R, Yu G (2021) User association for millimeter-wave ultra-reliable low-latency communications. IEEE Wireless Commun Lett 10(2):315–319

    Article  Google Scholar 

  • Liu D, Wang L, Chen Y, Elkashlan M, Wong K, Schober R, Hanzo L (2016) User association in 5G networks: a survey and an outlook. IEEE Commun Surveys Tuts 18(2):1018–1044

    Article  Google Scholar 

  • Liu R, Lee M, Yu G, Li GY (2020) User association for millimeter-wave networks: a machine learning approach. IEEE Trans Commun 68(7):4162–4174

    Article  Google Scholar 

  • Mohamed EM, Sakaguchi K, Sampei S (2017) Wi-Fi coordinated WiGig concurrent transmissions in random access scenarios. IEEE Trans Veh Technol 66(11):10357–10371

    Article  Google Scholar 

  • Mohamed EM, Abdelghany MA, Zareei M (2019) An efficient paradigm for multiband WiGig D2D networks. IEEE Access 7:70032–70045

    Article  Google Scholar 

  • Mohamed EM, ElHalawany BM, Khallaf HS, Zareei M, Zeb A, Abdelghany MA (2020a) Relay probing for millimeter wave multi-hop D2D networks. IEEE ACCESS 8:30560–30574

    Article  Google Scholar 

  • Mohamed EM, Hashima S, Aldosary A, Hatano K, Abdelghany MA (2020b) Gateway selection in millimeter wave UAV wireless networks using multi-player multi-armed bandit. Sensors 20(14):1–22

    Article  Google Scholar 

  • Oz̈koç MF, Koutsaftis A, Kumar R, Liu P, Panwar SS (2021) The impact of multi-connectivity and handover constraints on millimeter wave and terahertz cellular networks. IEEE J Select Areas Commun 39(6):1833–1853

    Article  Google Scholar 

  • Peng M, He G, Wang L, Kai C (2019) AP selection scheme based on achievable throughputs in SDN-enabled WLANs. IEEE Access 7:4763–4772

    Article  Google Scholar 

  • Rappaport TS, Sun S, Mayzus R, Zhao H et al (2013a) Millimeter wave mobile communications for 5G cellular: it will work! IEEE Access 1:335–349

    Article  Google Scholar 

  • Rappaport TS, Gutierrez F, Ben-Dor E et al (2013b) Broadband millimeter-wave propagation measurements and models using adaptive-beam antennas for outdoor urban cellular communications. IEEE Trans Antenn Propag 61(4):850–1859

    Article  Google Scholar 

  • Sakaguchi K, Mohamed EM, Kusano H et al (2015) Millimeter-wave wireless LAN and its extension toward 5G heterogeneous networks. IEICE Trans Commun E98.B(10):1932–1948

    Article  Google Scholar 

  • Seldin Y, Szepesvari C, Auer P, Yadkori Y A (2012) Evaluation and analysis of the performance of the EXP3 algorithm in stochastic environments. In: the 10th European Workshop on Reinforcement Learning: 103–116

  • Singh S, Kulkarni MN, Ghosh A, Andrews JG (2015) Tractable model for rate in self-backhauled millimeter wave cellular networks. IEEE J Selec Areas Commun 33(10):2196–2211

    Article  Google Scholar 

  • Walsh TJ, Szita I, Diuk C, Littman ML (2009) Exploring compact reinforcement-learning representations with linear regression. In: the 25th Conference on Uncertainty in Artificial Intelligence: 1–11

  • Wu S, Atat R, Mastronarde N, Liu L (2018) Improving the coverage and spectral efficiency of millimeter-wave cellular networks using device-to-device relays. IEEE Transc Commun 66(5):2251–2265

    Article  Google Scholar 

  • Yoo S K, Zhang L, Cotton S L, Quoc Ngo H (2019) Distributed antenna systems used for indoor UE to access point communications at 60 GHz. In: the 13th European Conference on Antennas and Propagation (EuCAP): 1–5

  • Zhang X, Zhou S, Wang X et al (2012) Improving network throughput in 60GHz WLANs via multi-AP diversity. In: 2012 IEEE International Conference on Communications (ICC): 4803–4807

  • Zhang L, Cotton S L, Yoo S, Fernández M, Scanlon W G (2021) Access point selection strategies for indoor 5G millimeter-wave distributed antenna systems. In: the 5th European Conference on Antennas and Propagation (EuCAP): 1–5

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, College of Engineering, Prince Sattam Bin Abdulaziz University, Wadi Ad Dawasir, 11991, Saudi Arabia

    Ehab Mahmoud Mohamed

  2. Electrical Engineering Department, Faculty of Engineering, Aswan University, Aswan, 81542, Egypt

    Ehab Mahmoud Mohamed

Authors
  1. Ehab Mahmoud Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ehab Mahmoud Mohamed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohamed, E.M. WiGig access point selection using non-contextual and contextual multi-armed bandit in indoor environment. J Ambient Intell Human Comput 14, 11833–11848 (2023). https://doi.org/10.1007/s12652-022-03739-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-03739-7

Keywords

Search

Navigation