Abstract
Wireless communications play an important part in the systems of the Internet of Things (IoT). Recently, there has been a trend towards long-range communications systems for IoT, including cellular networks. For many use cases, such as massive machine-type communications (mMTC), performance can be gained by going out of the classical model of connection establishment and adopting the random access methods. Associated with physical layer techniques such as Successive Interference Cancellation (SIC), or Non-Orthogonal Multiple Access (NOMA), the performance of random access can be dramatically improved, giving the novel random access protocol designs. This article studies one of these modern random access protocols: Irregular Repetition Slotted Aloha (IRSA). Because optimizing its parameters is not an easily solved problem, in this article, we use a reinforcement learning approach for that purpose. We adopt one specific variant of reinforcement learning, Regret Minimization, to learn the protocol parameters. We explain why it is selected, how to apply it to our problem with centralized learning, and finally, we provide both simulation results and insights into the learning process. The obtained results show the excellent performance of IRSA when it is optimized with Regret Minimization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Note that then the learning also behaves as if one class would be one agent by itself. The algorithm, and our implementation, also works with non-shared tables.
- 2.
It is indeed possible to construct a frameless IRSA scenario where one user can be only decoded after an arbitrarily large delay.
References
Bloembergen, D., Tuyls, K., Hennes, D., Kaisers, M.: Evolutionary dynamics of multi-agent learning: a survey. JAIR 53, 659–697 (2015)
Blum, A., Mansour, Y.: Learning, regret minimization, and equilibria. In: Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V.V. (eds.) Algorithmic Game Theory, pp. 79–102. Cambridge University Press (2007)
Chu, Y., Mitchell, P.D., Grace, D.: ALOHA and Q-Learning based medium access control for Wireless Sensor Networks. In: Proceedings of ISWCS 2012, pp. 511–515, August 2012. ISSN 2154–0225
Liva, G.: Graph-based analysis and optimization of contention resolution diversity slotted ALOHA. IEEE Trans. Commun. 59(2), 477–487 (2011)
Paolini, E., Liva, G., Chiani, M.: Coded slotted ALOHA: a graph-based method for uncoordinated multiple access. IEEE Trans. Inform. Theory 61(12), 6815–6832 (2015)
Srivatsa, C.R., Murthy, C.R.: Throughput analysis of PDMA/IRSA under practical channel estimation. In: 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), pp. 1–5, July 2019. ISSN 1948–3252
Stefanovic, C., Popovski, P., Vukobratovic, D.: Frameless ALOHA protocol for wireless networks. IEEE Commun. Lett. 16(12), 2087–2090 (2012)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series, 2nd edn. The MIT Press, Cambridge (2018)
Wang, S., Liu, H., Gomes, P.H., Krishnamachari, B.: Deep reinforcement learning for dynamic multichannel access in wireless networks. IEEE Trans. Cogn. Commun. Netw. 4(2), 257–265 (2018)
Zinkevich, M., Johanson, M., Bowling, M., Piccione, C.: Regret Minimization in Games with Incomplete Information, page 8
Casini, E., De Gaudenzi, R., Del Rio Herrero, O.: Contention resolution diversity slotted ALOHA (CRDSA): an enhanced random access scheme for satellite access packet networks. IEEE Trans. Wirel. Commun. 6(4), 1408–1419 (2007)
Fooladivanda, D., Al Daoud, A., Rosenberg, C.: Joint resource allocation and user association for heterogeneous wireless cellular networks. IEEE Trans. Wirel. Commun. 12, 384–390 (2011)
Ge, X., Li, X., Jin, H., Cheng, J., Leung, V.C.M.: Joint user association and user scheduling for load balancing in heterogeneous networks. IEEE Trans. Wirel. Commun. 17, 3211–3225 (2018)
Luong, N.C., et al.: Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21(4), 3133–3174 (2019)
Naparstek, O., Cohen, K.: Deep multi-user reinforcement learning for distributed dynamic spectrum access. IEEE Trans. Wirel. Commun. 18, 310–323 (2019)
Destounis, A., Tsilimantos, D., Debbah, M., Paschos, G.S.: Learn2MAC: online learning multiple access for URLLC applications. In: IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Paris, France, pp. 1–6 (2019)
Toni, L., Frossard, P.: IRSA Transmission Optimization via Online Learning (2018)
Wang, L., Xiao J., Guanrong, C.: Density evolution method and threshold decision for irregular LDPC codes. In: International Conference on Communications, Circuits and Systems (IEEE Cat. No.04EX914), Chengdu, vol. 1, pp. 25–28 (2004). https://doi.org/10.1109/ICCCAS.2004.1345932
Hmedoush, I., Adjih, C., Mühlethaler, P., Kumar, V.: On the performance of irregular repetition slotted aloha with multiple packet reception. In: International Wireless Communications and Mobile Computing (IWCMC), Limassol, Cyprus 2020, pp. 557–564 (2020). https://doi.org/10.1109/IWCMC48107.2020.9148173
Nguyen, T.T., Nguyen, N.D., Nahavandi, S.: Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications. IEEE Trans. Cybern. 50(9), 3826–3839 (2020)
Klos, T., van Ahee, G.J., Tuyls, K.: Evolutionary dynamics of regret minimization. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6322, pp. 82–96. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15883-4_6
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Hmedoush, I., Adjih, C., Mühlethaler, P. (2021). A Regret Minimization Approach to Frameless Irregular Repetition Slotted Aloha: IRSA-RM. In: Renault, É., Boumerdassi, S., Mühlethaler, P. (eds) Machine Learning for Networking. MLN 2020. Lecture Notes in Computer Science(), vol 12629. Springer, Cham. https://doi.org/10.1007/978-3-030-70866-5_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-70866-5_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-70865-8
Online ISBN: 978-3-030-70866-5
eBook Packages: Computer ScienceComputer Science (R0)