Abstract
The digital system of the future will face the growing challenge of controlling the system behavior in complex dynamically evolving environments. In this paper, we examine the applicability of a new management paradigm based on a reinforcement learning approach, where no preliminary specification of the system model is required. The learning agent identifies the most adequate control policies in live interaction with a partially observed system and provides it with autonomous management capabilities. We present the results of experimentation with cloud-based applications and discuss the technical challenges that need to be addressed in this field. Furthermore, we present the results of experimentation on a 5G network slice that hosts a cloud-based application in a multi-agent reinforcement learning setting, and demonstrate the value of information exchange between learning agents.
Similar content being viewed by others
References
Sutton RS, Barto AG (1992) Reinforcement learning: an introduction, 2nd edn. The MIT Press, Cambridge, Massachusetts; London, England
Clark J (2015) This preschool is for robots. Bloomberg
Gu S, Holly E, Lillicrap T, Levine S (2017) Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. In: IEEE international conference on robotics and automation (ICRA), Singapore
Pit.ai, https://www.pit.ai/
Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J et al (2015) Human-level control through deep reinforcement learning. Nature 518:529–533
Silver D, Hassabis D (2016) AlphaGo: mastering the ancient game of go with machine learning. Google Research Blog
Jamshidi P, Sharifloo A, Pahl C, Metzger A, Estrada G (2015) Self-learning cloud controllers: fuzzy q-learning for knowledge evolution. In: International conference on cloud and autonomic computing, pp 208–211
OpenStack Cloud Operating System, http://www.openstack.org/
Anicas M, An introduction to HAProxy and load balancing concepts. https://www.digitalocean.com/community/tutorials/an-introduction-to-haproxy-and-load-balancing-concepts
Erinle B (2015) Performance testing with JMeter, 2nd edn. Packt Publishing
Luong D-H, Thieu H-T, Outtagarts A, Ghamri-Doudane Y (2018) Predictive autoscaling orchestration for cloud-native telecom microservices. In: 2018 IEEE 5G world forum (5GWF). IEEE, pp 153–158
Auto scaling in Amazon web services, https://aws.amazon.com/autoscaling/
Jacobson D, Yuan D, Joshi N (2013) Scryer: Netflix’s predictive auto scaling engine. Netflix Technology Blog
Roy N, Dubey A, Gokhale A (2011) Efficient autoscaling in the cloud using predictive models for workload forecasting. In: Proceedings of the 2011 IEEE 4th international conference on CLOUD computing (CLOUD '11), Washington, DC, USA, pp 500–507
Li H and Venugopal S (2011) Using reinforcement learning for controlling an elastic web application hosting platform. International conference on automatic computing. pp 205–208
Rao J, Bu X, Xu C-Z and Wang K (2011) A distributed self-learning approach for elastic provisioning of virtualized cloud resources. 19th annual IEEE international symposium on modelling, analysis, and simulation of computer and telecommunication systems. pp 45–54
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jin, Y., Bouzid, M., Kostadinov, D. et al. Resource management of cloud-enabled systems using model-free reinforcement learning. Ann. Telecommun. 74, 625–636 (2019). https://doi.org/10.1007/s12243-019-00720-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12243-019-00720-y