Skip to main content
SpringerLink
Log in

Reinforcement Learning in Dynamic Task Scheduling: A Review

  • Review Article
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Scheduling is assigning shared resources over time to efficiently complete the tasks over a given period of time. The term is applied separately for tasks and resources correspondingly in task scheduling and resource allocation. Scheduling is a popular topic in operational management and computer science. Effective schedules ensure system efficiency, effective decision making, minimize resource wastage and cost, and enhance overall productivity. It is generally a tedious task to choose the most accurate resources in performing work items and schedules in both computing and business process execution. Especially in real-world dynamic systems where multiple agents involve in scheduling various dynamic tasks is a challenging issue. Reinforcement Learning is an emergent technology which has been able to solve the problem of the optimal task and resource scheduling dynamically. This review paper is about a research study that focused on Reinforcement Learning techniques that have been used for dynamic task scheduling. The paper addresses the results of the study by means of the state-of-the-art on Reinforcement learning techniques used in dynamic task scheduling and a comparative review of those techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of Data and Material

Not applicable.

References

  1. Zhang D, Han X, Deng C. Review on the research and practice of deep learning and reinforcement learning in smart grids. CSEE J Power Energy Syst. 2018;4(3):362–70. https://doi.org/10.17775/CSEEJPES.2018.00520.

    Article  Google Scholar 

  2. Xie J, Gao L, Peng K, Li X, Li H. Review on flexible job shop scheduling. IET Collab Intell Manuf. 2019;1(3):67–77. https://doi.org/10.1049/iet-cim.2018.0009.

    Article  Google Scholar 

  3. Luo S. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput. 2020;91:106208. https://doi.org/10.1016/j.asoc.2020.106208.

    Article  Google Scholar 

  4. Nashid Anjum MD, Wang H. Dynamic scheduling and analysis of real time systems with multiprocessors. Digit Commun Netw. 2016;2(3):130–8. https://doi.org/10.1016/j.dcan.2016.06.004.

    Article  Google Scholar 

  5. Hagras T, Janeček J. Static vs. dynamic list-scheduling performance comparison. Acta Polytechn. 2003;43(6):16–21.

    Google Scholar 

  6. Kopetz H, Real-time scheduling. In: Real-time systems. The International series in engineering and computer science. vol 395. Springer: Boston; 2002.

  7. Huang Z, van der Aalst WMP, Lu X, Duan H. Reinforcement learning based resource allocation in business process management. Data Knowl Eng. 2011;70(1):127–45.

    Article  Google Scholar 

  8. Kumar V, Bhambri S, Shambharkar PG. Multiple resource management and burst time prediction using deep reinforcement learning. In: Eighth International Conference on advances in computing, communication and information technology CCIT, 2019, pp. 51–58.

  9. Xiao Z, Ma S, Zhang S. Learning task allocation for multiple flows in multi-agent systems. In: 2009 International Conference on communication software and networks, Chengdu Sichuan, China; 2009, pp. 153–157. https://doi.org/10.1109/ICCSN.2009.28.

  10. Zhao X, Zong Q, Tian B, Zhang B, You M. Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning. Aerosp Sci Technol. 2019;92:588–94. https://doi.org/10.1016/j.ast.2019.06.024.

    Article  Google Scholar 

  11. Nguyen H, La H. Review of Deep Reinforcement Learning for Robot Manipulation. In: 2019 Third IEEE International Conference on robotic computing (IRC). Naples, Italy; 2019, pp. 590–595. https://doi.org/10.1109/IRC.2019.00120.

  12. Hou J, Li H, Hu J, Zhao C, Guo Y, Li S et al. A review of the applications and hotspots of reinforcement learning. In: 2017 IEEE International Conference on unmanned systems (ICUS). Beijing, China; 2017, pp. 506–511. https://doi.org/10.1109/ICUS.2017.8278398.

  13. Zhang L, Qi Q, Wang J, Sun H, Liao J. Multi-task deep reinforcement learning for scalable parallel task scheduling. In: 2019 IEEE International Conference on big data (Big Data). Los Angeles, CA, USA: IEEE; 2019, pp. 2992–3001. https://doi.org/10.1109/BigData47090.2019.9006027.

  14. Sutton RS, Barto AG. Reinforcement learning: an introduction. 2nd ed. Cambridge: The MIT Press; 2018.

    MATH  Google Scholar 

  15. Tian Y-T, Yang M, Qi X-Y, Yang Y-M. Multi-robot task allocation for fire-disaster response based on reinforcement learning. In: 2009 International Conference on machine learning and cybernetics. IEEE; 2009, pp. 2312–2317. https://doi.org/10.1109/ICMLC.2009.5212216.

  16. Arel I, Liu C, Urbanik T, Kohls AG. Reinforcement learning-based multi-agent system for network traffic signal control. IET Intell Transport Syst. 2010;4(2):128. https://doi.org/10.1049/iet-its.2009.0070.

    Article  Google Scholar 

  17. Wang Y-C, Usher JM. Application of reinforcement learning for agent-based production scheduling. Eng Appl Artif Intell. 2005;18(1):73–82. https://doi.org/10.1016/j.engappai.2004.08.018.

    Article  Google Scholar 

  18. Sun Y, Tan W. A trust-aware task allocation method using deep q-learning for uncertain mobile crowdsourcing. Hum Cent Comput Inf Sci. 2019;9(1):25. https://doi.org/10.1186/s13673-019-0187-4.

    Article  Google Scholar 

  19. Ben Noureddine D, Gharbi A, Ben Ahmed S. Multi-agent Deep Reinforcement Learning for Task Allocation in Dynamic Environment: In: Proceedings of the 12th International Conference on software technologies, Madrid, Spain: SCITEPRESS–Science and Technology Publications; 2017. p. 17-26. https://doi.org/10.5220/0006393400170026.

  20. Zhang K, Zhu Y, Leng S, He Y, Maharjan S, Zhang Y. Deep learning empowered task offloading for mobile edge computing in urban informatics. IEEE Internet Things J. 2019;6(5):7635–47. https://doi.org/10.1109/JIOT.2019.2903191.

    Article  Google Scholar 

  21. Chantaravarapan S, Gunal A, Williams EJ. On Using Monte Carlo Methods for Scheduling. In: Proceedings of the 2004 Winter Simulation Conference, 2004. Washington, D.C.: IEEE; 2004, pp. 789–794. https://doi.org/10.1109/WSC.2004.1371542.

  22. Zhang W, Dietterich TG. A Reinforcement Learning Approach to Job-shop Scheduling. In: Proceedings of the 14th International Joint Conference on artificial intelligence (IJCAI-95). Morgan Kaufmann, Orlando, FL; 1995, pp. 1114–1120.

  23. Zhang W. Reinforcement learning for job-shop scheduling [Doctor of Philosophy in Computer Science]. Oregon State University; 1996.

  24. Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I. Multi-agent actor-critic for mixed cooperative-competitive environments. arXiv:170602275 [cs] [Internet]. 2017 Jun 7; Available from: http://arxiv.org/abs/1706.02275.

  25. Wei Y, Yu FR, Song M, Han Z. User scheduling and resource allocation in HetNets with hybrid energy supply: an actor-critic reinforcement learning approach. IEEE Trans Wirel Commun. 2018;17(1):680–92. https://doi.org/10.1109/TWC.2017.2769644.

    Article  Google Scholar 

  26. Liu C-L, Chang C-C, Tseng C-J. Actor-critic deep reinforcement learning for solving job shop scheduling problems. IEEE Access. 2020;8:71752–62. https://doi.org/10.1109/ACCESS.2020.2987820.

    Article  Google Scholar 

  27. Kim D, Moon S, Hostallero D, Kang WJ, Lee T, Son K, et al. Learning to schedule communication in multi-agent reinforcement learning. arXiv:190201554 [cs] [Internet]. 2019 Feb 5; Available from: http://arxiv.org/abs/1902.01554.

  28. Wiering M. Multi-agent reinforcement learning for traffic light control. In: 17th International Conf. on Machine Learning (ICML). 2000; pp. 1151–1158.

  29. Sutton RS. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Machine learning proceedings 1990. Elsevier; 1990. pp. 216–224. https://doi.org/10.1016/B978-1-55860-141-3.50030-4.

  30. Peng B, Li X, Gao J, Liu J, Wong K-F. Deep Dyna-Q: integrating planning for task-completion dialogue policy learning. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Melbourne, Australia: Association for Computational Linguistics; 2018. pp. 2182–2192. https://doi.org/10.18653/v1/P18-1203.

  31. Su S, Li X, Gao J, Liu J, Chen Y. Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. https://arxiv.org/abs/1808.09442.

  32. Peng J, Williams RJ. Efficient learning and planning within the dyna framework. 7. https://doi.org/10.1177/105971239300100403.

  33. Cui J, Liu Y, Nallanathan A. Multi-agent reinforcement learning based resource allocation for UAV networks. IEEE Trans Wirel Commun. 2019;. https://doi.org/10.1109/TWC.2019.2935201.

    Article  Google Scholar 

  34. Zheng L, Yang J, Cai H, Zhang W, Wang J, Yu Y. MAgent: a many-agent reinforcement learning platform for artificial collective intelligence. arXiv:171200600 [cs] [Internet]. 2017 Dec 2; Available from: http://arxiv.org/abs/1712.00600.

  35. Gabel T, Riedmiller M. Adaptive reactive job-shop scheduling with reinforcement learning agents. Int J Inf Technol Intell Comput. 2008a;24(4):30.

    Google Scholar 

  36. Wu J, Xu X, Zhang P, Liu C. A novel multi-agent reinforcement learning approach for job scheduling in Grid computing. Future Gener Comput Syst. 2011;27(5):430–9. https://doi.org/10.1016/j.future.2010.10.009.

    Article  Google Scholar 

  37. Wu J, Xu X. Decentralised grid scheduling approach based on multi-agent reinforcement learning and gossip mechanism. CAAI Trans Intell Technol. 2018;3(1):8–17. https://doi.org/10.1049/trit.2018.0001.

    Article  Google Scholar 

  38. Moradi MA: Centralized reinforcement learning method for multi-agent job scheduling in Grid. In: 6th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Iran: IEEE; 2016. pp. 171–6. https://doi.org/10.1109/ICCKE.2016.7802135.

Download references

Acknowledgements

The corresponding author wish to express gratitude Dr. Thushari Silva and Professor Asoka Karunananda for their massive guidance and commitment throughout the research.

Funding

The funding is handled by the Authors itself.

Author information

Authors and Affiliations

  1. Department of Computational Mathematics, Faculty of Information Technology, University of Moratuwa, Katubedda, Sri Lanka

    Chathurangi Shyalika, Thushari Silva & Asoka Karunananda

Authors
  1. Chathurangi Shyalika
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thushari Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asoka Karunananda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chathurangi Shyalika.

Ethics declarations

Conflicts of Interest/Competing Interests

The authors declare that there are no conflicts of interest regarding the publication of this article.

Code Availability

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shyalika, C., Silva, T. & Karunananda, A. Reinforcement Learning in Dynamic Task Scheduling: A Review. SN COMPUT. SCI. 1, 306 (2020). https://doi.org/10.1007/s42979-020-00326-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-020-00326-5

Keywords

Search

Navigation