Abstract
In this paper, we investigate a deep Q-network (DQN)-based method for applying a dynamic spectrum access model to device-to-device (D2D) communications underlying cellular networks. Dynamic spectrum access (DSA) devices have two critical concerns, namely avoiding interference to primary users (PUs) and interference coordination with other secondary users (SUs). We consider that the issues faced by DSA users are also applicable to the D2D communication underlying cellular network. Therefore, we propose a distributed dynamic spectrum access scheme based on deep reinforcement learning (DRL). It enables each D2D user to learn a reliable spectrum access policy through imperfect spectrum sensing without knowledge of system prior information, avoiding collisions with cellular users and other D2D users and maximizing system throughput. Finally, the simulation results demonstrate the effectiveness of our proposed dynamic spectrum access scheme.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Forecast, G., et al.: Cisco visual networking index: global mobile data traffic forecast update, 2017–2022. Update 2017, 2022 (2019)
Yin, S., Chen, D., Zhang, Q., Liu, M., Li, S.: Mining spectrum usage data: a large-scale spectrum measurement study. IEEE Trans. Mob. Comput. 11(6), 1033–1046 (2012)
McHenry, M.A., Tenhula, P.A., McCloskey, D., Roberson, D.A., Hood, C.S.: Chicago spectrum occupancy measurements & analysis and a long-term studies proposal. In: Proceedings of the first International Workshop on Technology and Policy for Accessing Spectrum, August 2006
Asadi, A., Wang, Q., Mancuso, V.: A survey on device-to-device communication in cellular networks. IEEE Commun. Surv. Tutorials 16(4), 1801–1819 (2014)
Ansari, R.I., Chrysostomou, C., Hassan, S.A., Guizani, M., Mumtaz, S., Rodriguez, J., Rodrigues, J.J.: 5g d2d networks: techniques, challenges, and future prospects. IEEE Syst. J. 12(4), 3970–3984 (2018)
Kolodzy, P., Avoidance, I.: Spectrum policy task force. Federal Commun. Comm. Washington, DC, Rep. ET Docket 40(4), 147–158 (2002)
Shah, S.W.H., Mian, A.N., Crowcroft, J.: Statistical qos guarantees for licensed-unlicensed spectrum interoperable d2d communication. IEEE Access 8, 27 277–27 290 (2020)
Kai, Y., Wang, J., Zhu, H., Wang, J.: Resource allocation and performance analysis of cellular-assisted OFDMA device-to-device communications. IEEE Trans. Wirel. Commun. 18(1), 416–431 (2019)
Song, H., Liu, L., Ashdown, J., Yi, Y.: A deep reinforcement learning framework for spectrum management in dynamic spectrum access. IEEE Internet Things J. 8(14), 11 208–11 218 (2021)
Zia, K., Javed, N., Sial, M.N., Ahmed, S., Pirzada, A.A., Pervez, F.: A distributed multi-agent RL-based autonomous spectrum allocation scheme in d2d enabled multi-tier hetnets. IEEE Access 7, 6733–6745 (2019)
Gong, P.-Y., Wang, C.-H, Sheu, J.-P., Yang, D.-N.: Distributed drl-based resource allocation for multicast d2d communications. In: 2021 IEEE Global Communications Conference (GLOBECOM), pp. 01–06. December 2021
Huang, J., Yang, Y., He, G., Xiao, Y., Liu, J.: Deep reinforcement learning-based dynamic spectrum access for d2d communication underlay cellular networks. IEEE Commun. Lett. 25(8), 2614–2618 (2021)
Chang, H.-H., Song, H., Yi, Y., Zhang, J., He, H., Liu, L.: Distributive dynamic spectrum access through deep reinforcement learning: a reservoir computing-based approach. IEEE Internet Things J. 6(2), 1938–1948 (2019)
Zhao, Q., Krishnamachari, B., Liu, K.: On myopic sensing for multi-channel opportunistic access: structure, optimality, and performance. IEEE Trans. Wirel. Commun. 7(12), 5431–5440 (2008)
Meinilä, J., Kyösti, P., Jämsä, T., Hentilä, L.: Winner ii channel models. In: Radio Technologies and Concepts for IMT-Advanced, February 2008
Acknowledgment
This work was supported by the National Natural Science Foundation of China (62001327, 61701345), Natural Science Foundation of Tianjin (18JCZDJC31900).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Jiang, Z., Han, L., Wang, X. (2023). Distributed Dynamic Spectrum Access for D2D Communications Underlying Cellular Networks Using Deep Reinforcement Learning. In: Liang, Q., Wang, W., Mu, J., Liu, X., Na, Z. (eds) Artificial Intelligence in China. AIC 2022. Lecture Notes in Electrical Engineering, vol 871. Springer, Singapore. https://doi.org/10.1007/978-981-99-1256-8_40
Download citation
DOI: https://doi.org/10.1007/978-981-99-1256-8_40
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-1255-1
Online ISBN: 978-981-99-1256-8
eBook Packages: Computer ScienceComputer Science (R0)