Abstract
Massive multiple-input multiple-output (MMIMO) is a WiFi access technique studied and investigated in response to the worldwide bandwidth bottleneck in the WiFi telecommunication industry. Massive MIMO, which brings multiple antennae to transmission and reception to deliver excellent spectrum and power effectiveness with comparatively simple computation, is among the leading fundamental technologies for next-generation networking. For such a practical implementation of 5G—and further, that networks will realize many implementations of the smart sensor device—it is essential to gain a greater understanding of such a massive MIMO model to address its underlying problems. Because of the significant achievements of reinforcement learning (RL) and deep learning (DL), new and potent techniques are now available to help MIMO telecommunication networks deal with problems. This paper presents a thorough analysis of the convergence among the two fields, emphasizing RL and DL methods for MIMO networks. Throughout this article, a framework for RL-based beam-forming vector assault defense has been presented (reinforcement learning). Its outcomes demonstrated acceptable efficiency as well as the anticipated outcome.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
No new data was generated for this project.
References
Aytar, Y.: Playing hard exploration games by watching youtube. In: Proceedings of the annual conference on neural information processing systems 2018, NeurIPS 2018, Montreal, QC, Canada (2018)
Brunner, G., Richter, O., Wang, Y., Wattenhofer, R.: Teaching a machine to read maps with deep reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence (AAAI), New Orleans, LA (2018)
Cao, Y., Zhang, G., Li, G., Zhang, J.: A deep Q-network based-resource allocation scheme for massive MIMO-NOMA. IEEE Commun. Lett. 25, 1544–1548 (2021)
Christopher, B., Greg, B., Brooke, C.: Dota 2 with large scale deep reinforcement learning. arXiv:1912.06680.
Coronato, A., Naeem, M., De Pietro, G., Paragliola, G.: Reinforcement learning for intelligent healthcare applications: a survey. Artif. Intell. Med. 109, 101964 (2020)
Gante, J., Falcao, G., Sousa, L.: Deep learning architectures for accurate millimeter wave positioning in 5G. Neural. Process. Lett. 51, 487–514 (2020)
He, H., Wen, C.K., Jin, S., Li, G.Y.: Deep learning-based channel estimation for beamspacemmWave massive MIMO systems. IEEE Wirel. Commun. Lett. 7, 852–855 (2018)
Jeerige, A., Bein, D., Verma, A.: Comparison of deep reinforcement learning approaches for intelligent game playing. In: Proceedings of the 9th annual computing and communication workshop and conference (CCWC), pp. 0366–0371, Las Vegas, NV (2019)
Künzel, G., Cainelli, G.P., Müller, I., Pereira, C.E.: Weight adjustments in a routing algorithm for wireless sensor and actuator networks using Q-learning. IFAC-PapersOnLine 51, 58–63 (2018)
Le, H.A., Van Chien, T., Nguyen, T.H., Choo, H., Nguyen, V.D.: Machine learning-based 5g-and-beyond channel estimation for MIMO-OFDM communication systems. Sensors 21, 4861 (2021)
Li, P., Jiang, Y., Li, W., Zheng, F. and You, X.: A cmdp-based approach for energy efficient power allocation in massive mimo systems. In: 2016 IEEE wireless communications and networking conference, pp. 1–6 (2016)
Liu, X., Qin, Z., Gao, Y.: Resource allocation for edge computing in IoT networks via reinforcement learning. In: Proceedings of the ICC 2019–2019 IEEE international conference on communications (ICC), pp. 1–6, Shanghai (2019)
Liu, Y.J., Tang, L., Tong, S., Chen, C.P., Li, D.J.: Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems. IEEE Trans. Neural Netw. Learn. Syst. 26, 165–176 (2014)
Maksymyuk, T., Gazda, J., Yaremko, O., Nevinskiy, D.: Deep learning based massive MIMO beamforming for 5G mobile network. In: Proceedings of the 2018 IEEE 4th international symposium on wireless systems within the international conferences on intelligent data acquisition and advanced computing systems (IDAACS-SWS), pp. 241–244, Dortmund, (2018)
Mao, Q., Hu, F., Hao, Q.: Deep learning for intelligent wireless networks: a comprehensive survey. IEEE Commun. Surv. Tutor. 20, 2595–2621 (2018)
Markkandan, S., Logeshwaran, R.,Venkateswaran, N.: Analysis of Precoder Decomposition Algorithms for MIMO System Design. IETE Journal of Research, 1–8 (2021) https://doi.org/10.1080/03772063.2021.1920848
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Naeem, M., Paragliola, G., Coronato, A.: A reinforcement learning and deep learning based intelligent system for the support of impaired patients in home treatment. Expert Syst. Appl. 168, 114285 (2021)
Patel, P.A. Millimeter wave positioning with deep learning. Master's Thesis, California State University, Sacramento, CA, USA, 2020.
Sabarinath, R., Markkandan, S.: “Complexity Analysis and Performance Comparison of Channel Decomposition methods used in MIMO Precoder Design”. International Journal of Applied Engineering Research (IJAER), 2015.
Saravanan, M., Kumar, P.S., Sharma, A.: IoT Enabled indoor autonomous mobile robot using CNN and Q-learning. In: Proceedings of the 2019 IEEE international conference on industry 4.0, artificial intelligence, and communications technology (IAICT), Bali, Indonesia, (2019)
Shao, K., Tang, Z., Zhu, Y., Li, N., Zhao, D.: A survey of deep reinforcement learning in video games. arXiv:1912.10944.
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction, 2nd edn. MIT Press, Cambridge (2017)
Venkata Lalitha Narla, Renu Kachhoria, M. Arun, Anandakumar Haldorai, D. Vijendra Babu, Bos Mathew Jos. IoT based energy efficient multipath power control for underwater sensor network. International Journal of Systems Assurance Engineering and Management. Published by Springer Verlag Online ISSN: 0976–4348. https://doi.org/10.1007/s13198-021-01560-7
Wang, Q., Feng, K.: Precodernet: hybrid beamforming for millimeter wave systems using deep reinforcement learning. arXiv:1907.13266.
Wang, X., Gursoy, M.C.: Multiagent double deep Q-learning for beamforming in mmWave MIMO networks. In: Proceedings of the 2020 IEEE 31st annual international symposium on personal, indoor and mobile radio communications, Helsinki, Finland, pp. 1–6 (2020)
Wang, Q., Feng, K., Li, X., Jin, S.: PrecoderNet: hybrid beamforming for millimeter wave systems with deep reinforcement learning. IEEE Wirel. Commun. Lett. 9, 1677–1681 (2020)
Xin, Y.: Machine learning and deep learning methods for cybersecurity. IEEE Access 6, 35365–35381 (2018)
Zhou, B., Cui, Y., Tao, M.: Stochastic content-centric multicast scheduling for cache-enabled heterogeneous cellular networks. IEEE Trans. Wirel. Commun. 15(9), 6284–6297 (2016)
Zimaglia, E.: Deep learning application to 5G physical layer for channel estimation and csi feedback improvement. Ph.D. Thesis, Politecnico di Torino, Torino, Italy, 2019.
Funding
This research received no specific Grant from funding agencies in the public, commercial, or not-for-profit sectors.
Author information
Authors and Affiliations
Contributions
RNP: Spearheaded the research, conceptualized the RL-based framework for beam-forming vector assault defense, and supervised the overall project. AS: Played a pivotal role in the literature review, mainly focusing on the convergence of RL and DL in MIMO networks. Contributed to the drafting and editing of the manuscript. SV: Assisted in developing the theoretical underpinning of the massive MIMO model and its implications for next-generation networking. Provided critical revisions to the paper. MM: Led the data analysis and interpretation of results. Assisted Dr. Paranthaman in conceptualizing the RL-based approach. KVDS: Oversaw the simulation and validation of the proposed framework. Worked in tandem with M. Malathi to evaluate the efficiency of the model. MM: KVDS in simulations and validation processes. Additionally, provided insights into the real-world applicability and limitations of the presented framework.
Corresponding author
Ethics declarations
Conflict of interest
Author declares that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors. The work that has been submitted for publication does not include any financial or non-financial interests, the authors would want to make that clear.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Paranthaman, R.N., Sonker, A., Varalakshmi, S. et al. Reinforcement learning-based model for the prevention of beam-forming vector attacks on massive MIMO system. Opt Quant Electron 56, 44 (2024). https://doi.org/10.1007/s11082-023-05660-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11082-023-05660-5