Abstract
To ensure the reliable communication in mobile ad hoc networks (MANETs) with highly dynamic environment, this paper investigates cross-layer routing problem to minimize the system average packet delivery delay. Firstly, we construct a multi-agent cross-layer routing framework, where each node learns its routing policy cooperatively based on local observations. Secondly, we construct a decentralized partially observable Markov Decision Process (Dec-POMDP) by modelling the cross-layer routing problem based upon cross-layer partially observable environmental information. Then, we utilize multi-agent framework and employ reinforcement learning (RL) method based on Deep Graph Neural network (DGN) to incorporate the observations of neighboring agents with the graph attention convolutional kernel, and use the method of experience replay (ER) and target network for network stabilization and model training. Simulation results show that our proposed algorithm can achieve a 46.6% improvement in cumulative reward compared to the baseline without utilizing DGN, and exhibit higher performance and enhanced stability than the baselines when the number of agents increases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ali, A., Tariq, S., Iqbal, M., et al.: Adaptive bitrate video transmission over cognitive radio networks using cross layer routing approach. IEEE Trans. Cogn. Commun. Netw. 6(3), 935–945 (2020)
Sheikh, S.M., Halima, N.B.: A cross-layer hybrid strategy for congestion avoidance and link quality routing in wireless multi-hop networks. In: International Conference on Advances in Big Data, Computing and Data Communication Systems, pp. 1–8. IEEE (2018)
Wisitpongphan, N., Bai, F., Mudalige, P., et al.: Routing in sparse vehicular ad hoc wireless networks. IEEE J. Sel. Areas Commun. 25(8), 1538–1556 (2007)
Rehman, S., Khan, M., Zia, T., et al.: Quality of service base cross layer routing protocol for VANETs. In: 2015 IEEE/ACIS 16th International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), pp. 1–6. IEEE (2015)
Karp, B., Kung, H.T.: GPSR: greedy perimeter stateless routing for wireless networks. In: Proceedings of the 6th Annual International Conference on Mobile Computing and Networking, pp. 243–254 (2000)
Yau, A., Komisarczuk, P., Teal, P.: Reinforcement learning for context awareness and intelligence in wireless networks: review, new features and open issues. J. Netw. Comput. Appl. 35(1), 253–267 (2012)
Zhou, X., Lin, Y., Tu, Y., et al.: Dynamic channel allocation for multi-UAVs: a deep reinforcement learning approach. In: 2019 IEEE Global Communications Conference (GLOBECOM), pp. 1–6. IEEE (2019)
Karmakar, R., Chattopadhyay, S., Chakraborty, S.: Dynamic link adaptation in IEEE 802.11 ac: a distributed learning based approach. In: Local Computer Networks, pp. 87–94. IEEE (2016)
Yue, X.F., Wu, L.J., Duan, W.W.: Multi-agent reinforcement learning with contribution-based assignment online routing in SDN. In: 19th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), vol. 123(1), pp. 975–1001. IEEE (2022)
Jafarzadeh, O., et al.: A novel protocol for routing in vehicular ad hoc network based on model-based reinforcement learning and fuzzy logic. Int. J. Inf. Commun. Technol. Res. 12(4), 10–25 (2020)
Tan, X., et al.: Cooperative multi-agent reinforcement-learning-based distributed dynamic spectrum access in cognitive radio networks. IEEE Internet Things J. 9(19), 19477–19488 (2022)
Gu, Y., She, C., Quan, Z., et al.: Graph neural networks for distributed power allocation in wireless networks: aggregation over-the-air. IEEE Trans. Wirel. Commun. (2023). Early access
He, Z., Wang, L., et al.: Resource allocation based on graph neural networks in vehicular communications. In: 2020 IEEE Global Communications Conference (GLOBECOM 2020), pp. 1–5 (2020)
Hado, V.H., Arthur, G., David, S.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 2094–2100. Phoenix (2016)
Acknowledgements
This work was supported in part by the National Undergraduate Training Program for Innovation and Entrepreneurship under Item number 202210288098.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, Y., Xie, W., Ding, Z., Yang, Q., Lin, Y., Zhang, Y. (2024). Multi-agent Graph Reinforcement Learning Based Cross-Layer Routing for Mobile Ad-Hoc Network. In: Dong, J., Zhang, L., Cheng, D. (eds) Proceedings of the 2nd International Conference on Internet of Things, Communication and Intelligent Technology. IoTCIT 2023. Lecture Notes in Electrical Engineering, vol 1197. Springer, Singapore. https://doi.org/10.1007/978-981-97-2757-5_38
Download citation
DOI: https://doi.org/10.1007/978-981-97-2757-5_38
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2756-8
Online ISBN: 978-981-97-2757-5
eBook Packages: Computer ScienceComputer Science (R0)