A Multi-object Optimization Cloud Workflow Scheduling Algorithm Based on Reinforcement Learning

  • Wu Jiahao
  • Peng Zhiping
  • Cui Delong
  • Li Qirui
  • He Jieguang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10955)


In this paper, for the problem of long task scheduling time and unbalanced system load in the task scheduling of cloud workflow. To minimize the task scheduling time and optimize load balancing as the scheduling goal, a Markov decision process model conforming to the cloud workflow environment is established. Based on this, a multi-objective optimization cloud workflow scheduling algorithm based on reinforcement learning is proposed. The algorithm combines Q_Learning features, adding a function with a weighted fitness value function in the Q_Learning reward function so that it can apply multi-objective optimization. The set of scheduling schemes is a Pareto optimal solution set, which can select the optimal scheduling scheme according to the user’s preference. Compared with other methods, this algorithm can reduce the execution time and optimize the system load. And this paper uses the real cloud workflow data to carry out the simulation experiment, and carries on the experiment through the simulation platform WorkflowSim. The result proves the effectiveness of this algorithm.


Cloud workflow Reinforcement learning Task scheduling Load balancing 



Fund Project: National Natural Science Foundation of China (61772145, 61672174, 61272382), Guangdong Province Science and Technology Plan Project (2015B020233019, 2014A020208139).


  1. 1.
    Zuo, L.Y., Cao, Z.B.: An overview of research on scheduling problems in cloud computing. Appl. Res. Comput. 11(29), 4023–4027 (2012)Google Scholar
  2. 2.
    Lin, W.W., Qi, D.X.: Survey of cloud computing resource scheduling. Comput. Sci. 10(39), 1–6 (2012)Google Scholar
  3. 3.
    Topcuoglu, H., Hariri, S., Wu, M.Y., et al.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260–274 (2002)CrossRefGoogle Scholar
  4. 4.
    El-Rewini, H., Lewis, T.G.: Scheduling parallel program tasks onto arbitrary target machines. J. Parallel Distrib. Comput. 9(2), 138–153 (1990)CrossRefGoogle Scholar
  5. 5.
    Salza, P., Ferrucci, F., Sarro, F., et al.: Deploy and execute parallel genetic algorithms in the cloud. In: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pp. 121–122. ACM, Denver (2016)Google Scholar
  6. 6.
    Li, H.H., Chen, Z.G., Zhan, Z.H.: Renumber coevolutionary multiswarm particle swarm optimization for multi-objective workflow scheduling on cloud computing environment. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1419–1420. ACM, Madrid (2015)Google Scholar
  7. 7.
    Peng, Z.P., Cui, D.L., Zuo, J.L., et al.: Random task scheduling scheme based on reinforcement learning in cloud computing. Cluster Comput. 18(4), 1595–1607 (2015)CrossRefGoogle Scholar
  8. 8.
    Tian, G.Z., Xiao, C., Xie, J.Q.: Scheduled multi-DAG shared resource scheduling and fair cost optimization method. Chin. J. Comput. 37(7), 1607–1619 (2014)Google Scholar
  9. 9.
    Guo, T., Chen, Z., Yu, Y.L.: Workflow cost optimization model and algorithm for DAG with communication cost. J. Comput. Res. Dev. 52(6), 1400–1408 (2015)Google Scholar
  10. 10.
    Li, X.: Cloud computing task scheduling, computer measurement and control based on dependent task and saras(λ) algorithm. 23(8), 2809–2813 (2015)Google Scholar
  11. 11.
    Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)zbMATHGoogle Scholar
  12. 12.
    Hong, L.: Research on Workflow Scheduling Algorithm Based on Multi-objective Particle Swarm Optimization in Cloud Environment. Beijing Jiaotong University, Beijing (2015)Google Scholar
  13. 13.
    Chen, W., Deelman, E.: WorkflowSim: a toolkit for simulating scientific workflows in distributed environments. In: Proceedings of the IEEE International Conference on E-Science, pp. 1–8. IEEE, Bangalore (2012)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Wu Jiahao
    • 1
  • Peng Zhiping
    • 2
  • Cui Delong
    • 2
  • Li Qirui
    • 2
  • He Jieguang
    • 2
  1. 1.Department of ComputerGuangdong University of TechnologyGuangzhouChina
  2. 2.Department of Computer and Electronic InformationGuangdong University of Petrochemical TechnologyMaomingChina

Personalised recommendations