Deep Reinforcement Learning for Multi-resource Cloud Job Scheduling

  • Jianpeng Lin
  • Zhiping Peng
  • Delong Cui
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11303)


The resource scheduling problem in the cloud environment has always been a difficult and hot research field of cloud computing. The difficult problem of online decision-making tasks for resource management in a complex cloud environment can be solved by combining the excellent decision-making ability of reinforcement learning and the strong environmental awareness ability of deep learning. This paper proposes a multi-resource cloud job scheduling strategy in cloud environment based on Deep Q-network algorithm to minimize the average job completion time and average job slowdown. The experimental results show that the scheduling strategy is better than the scheduling strategy based on the standard policy gradient algorithm, and accelerate the convergence speed.


Cloud computing Deep reinforcement learning Job scheduling 



The work presented in this paper was supported by National Natural Science Foundation of China (61772145, 61672174).


  1. 1.
    Wang, T., Liu, Z., Chen, Y., Xu, Y., Dai, X.: Load balancing task scheduling based on genetic algorithm in cloud computing. In: Proceedings of the 12th International Conference on Dependable, Autonomic and Secure Computing, pp. 146–152 (2014)Google Scholar
  2. 2.
    Singh, S., Chana, I.: Resource provisioning and scheduling in clouds. QoS perspective. J. Supercomput. 72, 926–960 (2016)CrossRefGoogle Scholar
  3. 3.
    Zuo, L., Shu, L., Dong, S.: A multi-objective optimization scheduling method based on the ant colony algorithm in cloud computing. IEEE Access 3, 2687–2699 (2017)CrossRefGoogle Scholar
  4. 4.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  5. 5.
    Dutreilh, X., Kirgizov, S., Melekhova, O.: Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow, pp. 67–74 (2011)Google Scholar
  6. 6.
    Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput. Pract. Exp. 25, 1656–1674 (2013)CrossRefGoogle Scholar
  7. 7.
    Galstyan, A., Czajkowski, K., Lerman, K.: Resource allocation in the grid using reinforcement learning. In: International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1314–1315 (2004)Google Scholar
  8. 8.
    Peng, Z., Cui, D., Zuo, J.: Random task scheduling scheme based on reinforcement learning in cloud computing. Cluster Comput. 18, 1595–1607 (2015)CrossRefGoogle Scholar
  9. 9.
    Peng, Z., Cui, D., Zuo, J.: Research on cloud computing resources provisioning based on reinforcement learning. Math. Prob. Eng. 2015, 1–12 (2015)Google Scholar
  10. 10.
    Peng, Z., Cui, D., Ma, Y., Xiong, J., Xu, B., Lin, W.: A reinforcement learning-based mixed job scheduler scheme for cloud computing under SLA constraint. In: International Conference on Cyber Security and Cloud Computing, pp. 142–147 (2016)Google Scholar
  11. 11.
    Mnih, V., Kavukcuoglu, K., Silver, D.: Human-level control through deep reinforcement learning. Nature 518, 529 (2015)CrossRefGoogle Scholar
  12. 12.
    Mao, H., Alizadeh, M., Menache, I.: Resource management with deep reinforcement learning. In: ACM Workshop on Hot Topics in Networks, pp. 50–56 (2016)Google Scholar
  13. 13.
    Mnih, V., Kavukcuoglu, K., Silver, D.: Playing Atari with deep reinforcement learning. Computer Science (2013)Google Scholar
  14. 14.
    Hinton, G.: Overview of mini-batch gradient descent. Neural Networks for Machine Learning. Accessed 13 June 2018
  15. 15.
    Schulman, J., Levine, S., Moritz, P.: Trust region policy optimization. In: Computer Science, pp. 1889–1897 (2015)Google Scholar
  16. 16.
    Grandl, R., Ananthanarayanan, G., Kandula, S.: Multi-resource packing for cluster schedulers. ACM Sigcomm Comput. Commun. Rev. 44(4), 455–466 (2014)CrossRefGoogle Scholar
  17. 17.
    Liu, Q., Zhai, J.W., Zhang, Z.Z.: A survey on deep reinforcement learning. Chin. J. Comput. 40, 1–28 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Guangdong University of TechnologyGuangzhouChina
  2. 2.College of Computer and Electronic InformationGuangdong University of Petrochemical TechnologyMaomingChina

Personalised recommendations