Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement learning-based joint self-optimisation method for the fuzzy logic handover algorithm in 5G HetNets

  • S.I. : Neuro, fuzzy and their Hybridization
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

5G heterogeneous networks (HetNets) can provide higher network coverage and system capacity to the user by deploying massive small base stations (BSs) within the 4G macrosystem. However, the large-scale deployment of small BSs significantly increases the complexity and workload of network maintenance and optimisation. The current handover (HO) triggering mechanism A3 event was designed only for mobility management in the macrosystem. Directly implementing A3 in 5G-HetNets may degrade the user mobility robustness. Motivated by the concept of self-organisation networks (SON), this study developed a self-optimisation triggering mechanism to enable automated network maintenance and enhance user mobility robustness in 5G-HetNets. The proposed method integrates the advantages of subtractive clustering and Q-learning frameworks into the conventional fuzzy logic-based HO algorithm (FLHA). Subtractive clustering is first adopted to generate a membership function (MF) for the FLHA to enable FLHA with the self-configuration feature. Subsequently, Q-learning is utilised to learn the optimal HO policy from the environment as fuzzy rules that empower the FLHA with a self-optimisation function. The FLHA with SON functionality also overcomes the limitations of the conventional FLHA that must rely heavily on professional experience to design. The simulation results show that the proposed self-optimisation FLHA can effectively generate MF and fuzzy rules for the FLHA. The proposed approach can minimise the HO, ping-pong HO, and HO failure ratios while improving network throughput and latency by comparing with conventional triggering mechanisms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Shayea I, Ergen M, Hadri Azmi M, Aldirmaz Colak S, Nordin R, Daradkeh YI (2020) Key challenges, drivers and solutions for mobility management in 5G networks: a survey. IEEE Access 8:172534–172552

    Article  Google Scholar 

  2. Tayyab M, Gelabert X, Jantti R (2019) A survey on handover management: from LTE to NR. IEEE Access 7(1):118907–118930

    Article  Google Scholar 

  3. the 3GPP Organizational Partners, Telecommunication management; Self-Organising Networks (SON) Policy Network Resource Model (NRM) Integration Reference Point (IRP); Information Service (IS); document TS 28.628. 2020.

  4. the 3GPP Organizational Partners, Telecommunication management; Self-Organising Networks (SON); Concepts and requirements; document TS 32.500. 2020.

  5. the 3GPP Organizational Partners, Telecommunication management; Study on the Self-Organising Networks (SON) for 5G networks , document TR 28.861. 2019.

  6. the 3GPP Organizational Partners, Technical Specification Group Radio Access Network; NR; Radio Resource Control (RRC) protocol specification, document TS 38.331. 2020.

  7. Habbal A, Goudar SI, Hassan S (2017) Context-aware radio access technology selection in 5G ultra dense networks. IEEE Access 5:6636–6648

    Article  Google Scholar 

  8. Moysen J, Giupponi L (2018) From 4G to 5G: self-organised network management meets machine learning. Comput Commun 129:248–268

    Article  Google Scholar 

  9. Liu Q, Kwong CF, Wei S, Li L, Zhang S (2021) Intelligent handover triggering mechanism in 5G ultra-dense networks via clustering-based reinforcement learning. Mob Netw Appl 26(1):27–39

    Article  Google Scholar 

  10. Alhammadi A, Roslee M, Alias MY, Shayea I, Alraih S, Mohamed KS (2020) Auto tuning self-optimization algorithm for mobility management in LTE-A and 5G HetNets. IEEE Access 8:294–304

    Article  Google Scholar 

  11. Nguyen MT, Kwon S, Kim H (2018) Mobility robustness optimisation for handover failure reduction in LTE small-cell networks. IEEE Trans Veh Technol 67(5):4672–4676

    Article  Google Scholar 

  12. Hasan MM, Kwon S, Oh S (2019) Frequent-handover mitigation in ultra-dense heterogeneous networks. IEEE Trans Veh Technol 68(1):1035–1040

    Article  Google Scholar 

  13. Chaudhuri S, Baig I, Das D (2017) Self organising method for handover performance optimisation in LTE-advanced network. Comput Commun 110:151–163

    Article  Google Scholar 

  14. Goyal T, Kaushal S (2019) Handover optimisation scheme for LTE-advance networks based on AHP-TOPSIS and Q-learning. Comput Commun 133(2018):67–76

    Article  Google Scholar 

  15. Mwanje SS, Schmelz LC, Mitschele-Thiel A (2016) Cognitive cellular networks: a Q-learning framework for self-organising networks. IEEE Trans Netw Serv Manag 13(1):85–98

    Article  Google Scholar 

  16. Fakhfakh E, Hamouda S (2017) Optimised Q-learning for WiFi offloading in dense cellular networks. IET Commun 11(15):2380–2385

    Article  Google Scholar 

  17. M. Saeed, M. El-Ghoneimy, and H. Kamal, “An enhanced fuzzy logic optimisation technique based on user mobility for LTE handover,” in National Radio Science Conference, NRSC, Proceedings, 2017, pp. 230–237.

  18. Da Costa Silva K, Becvar Z, Frances CRL (2018) Adaptive hysteresis margin based on fuzzy logic for handover in mobile networks with dense small cells. IEEE Access 6:17178–17189

    Article  Google Scholar 

  19. J. Anand, A. S. Buttar, and R. Kaur (2018), “Fuzzy logic based spectrum handover approach in cognitive radio network: a Survey,” in Proceedings of the 2nd International Conference on Electronics, Communication and Aerospace Technology, ICECA 2018, Iceca, pp. 775–780.

  20. Aibinu AM, Onumanyi AJ, Adedigba AP, Ipinyomi M, Folorunso TA, Salami MJE (2017) Development of hybrid artificial intelligent based handover decision algorithm. Eng Sci Technol Int J 20(2):381–390

    Google Scholar 

  21. Vasudeva K, Dikmese S, Guven I, Mehbodniya A, Saad W, Adachi F (2017) Fuzzy-based game theoretic mobility management for energy efficient operation in HetNets. IEEE Access 5:7542–7552

    Article  Google Scholar 

  22. Ross TJ (2017) Fuzzy Logic with Engineering Applications. John Wiley & Sons Ltd, West Sussex, United Kingdom

    Google Scholar 

  23. Chen MS, Wang SW (1999) Fuzzy clustering analysis for optimising fuzzy membership functions. Fuzzy Sets Syst 103(2):239–254

    Article  Google Scholar 

  24. the 3GPP Organizational Partners, Study on channel model for frequencies from 0.5 to 100 GHz,document TR 38.901. 2019.

  25. Jiang F, Yuan Z, Sun C, Wang J (2019) Deep Q-learning-based content caching with update strategy for fog radio access networks. IEEE Access 7:97505–97514

    Article  Google Scholar 

  26. Su H, Zhang H, Gao DW, Luo Y (2020) Adaptive dynamics programming for H∞control of continuous-time unknown nonlinear systems via generalized fuzzy hyperbolic models. IEEE Trans Syst Man, Cybern Syst 50(11):3996–4008

    Article  Google Scholar 

  27. He S, Fang H, Zhang M, Liu F, Luan X, Ding Z (2019) Online policy iterative-based H∞ optimisation algorithm for a class of nonlinear systems. Inf Sci (Ny) 495:1–13

    Article  MATH  Google Scholar 

  28. He S, Fang H, Zhang M, Liu F, Ding Z (2020) Adaptive optimal control for a class of nonlinear systems: the online policy iteration approach. IEEE Trans Neural Netw Learn Syst 31(2):549–558

    Article  MathSciNet  Google Scholar 

  29. Su H, Zhang H, Jiang H, Wen Y (2020) Decentralised event-triggered adaptive control of discrete-time nonzero-sum games over wireless sensor-actuator networks with input constraints. IEEE Trans Neural Netw Learn Syst 31(10):4254–4266

    Article  MathSciNet  Google Scholar 

  30. Su H, Zhang H, Liang X, Liu C (2020) Decentralised event-triggered online adaptive control of unknown large-scale systems over wireless communication networks. IEEE Trans Neural Netw Learn Syst 31(11):4907–4919

    Article  MathSciNet  Google Scholar 

  31. He S, Zhang M, Fang H, Liu F, Luan X, Ding Z (2020) Reinforcement learning and adaptive optimisation of a class of markov jump systems with completely unknown dynamic information. Neural Comput Appl 32(18):14311–14320

    Article  Google Scholar 

  32. Yang D, Gao X, Cui E, Ma Z (2020) State-constraints adaptive backstepping control for active magnetic bearings with parameters nonstationarities and uncertainties. IEEE Trans Ind Electron 68(10):1–1

    Google Scholar 

Download references

Acknowledgements

The authors acknowledge financial support from the International Doctoral Innovation Centre (IDIC), Ningbo Education Bureau, Ningbo Science and Technology Bureau, and the University of Nottingham.

Author information

Authors and Affiliations

  1. International Doctoral Innovation Centre, University of Nottingham Ningbo China, Ningbo, China

    Qianyu Liu

  2. Department of Electrical and Electronic Engineering, University of Nottingham Ningbo China, Ningbo, China

    Chiew Foong Kwong & Lincan Li

  3. Department of Electrical and Electronic Engineering, University of Nottingham, Nottingham, UK

    Sun Wei & Sijia Zhou

  4. School of Computer Science, University of Nottingham Ningbo China, Ningbo, China

    Pushpendu Kar

Authors
  1. Qianyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chiew Foong Kwong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sijia Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lincan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pushpendu Kar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chiew Foong Kwong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Kwong, C.F., Wei, S. et al. Reinforcement learning-based joint self-optimisation method for the fuzzy logic handover algorithm in 5G HetNets. Neural Comput & Applic 35, 7297–7313 (2023). https://doi.org/10.1007/s00521-021-06673-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06673-5

Keywords

Search

Navigation