Skip to main content
SpringerLink
Log in

A self-learning discrete salp swarm algorithm based on deep reinforcement learning for dynamic job shop scheduling problem

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

For the dynamic job shop scheduling problem (DJSP), an improved variant of salp swarm algorithm (SSA) named self-learning discrete salp swarm algorithm (SLDSSA) is proposed to minimize makespan. The primary intentions are to enhance SSA’s exploration, exploitation, diversity, and dynamic balance of exploration and exploitation. SLDSSA benefits from three new improvement strategies: hybrid initialization, discrete position update strategy, and self-learning population partitioning mechanism. The hybrid initialization significantly improves the overall quality of the initial population. The proposed discrete update strategy enhances the exploration and exploitation capability of the algorithm. The self-learning population partitioning mechanism achieves a dynamic balance of exploration/exploitation rate according to the population state. The SlDSSA algorithm is tested on 25 test functions, 27 job shop benchmark instances , and composite DJSP instances to evaluate the performance of SLDSSA. Furthermore, the results of SLDSSA are compared with 13 existing algorithms. The results show that the SLDSSA algorithm can provide competitive results to the comparative algorithms, effectively solve job shop scheduling problems and deal with the interference caused by dynamic events.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Fig. 2
Algorithm 3
Fig. 3
Algorithm 4
Fig. 4
Algorithm 5
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Xiong H, Shi S, Ren D et al (2022) A survey of job shop scheduling problem: the types and models. Comput Oper Res 142. https://doi.org/10.1016/j.cor.2022.105731

  2. Prashar A, Tortorella GL, Fogliatto FS (2022) Production scheduling in industry 4.0: morphological analysis of the literature and future research agenda. J Manuf Syst 65:33–43. https://doi.org/10.1016/j.jmsy.2022.08.008

    Article  Google Scholar 

  3. Luo S (2020) Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning. Appl Soft Comput 91. https://doi.org/10.1016/j.asoc.2020.106208

  4. Li YF, He Y, Wang YL et al (2020) An optimization method for energy-conscious production in flexible machining job shops with dynamic job arrivals and machine breakdowns. J Clean Prod 254. https://doi.org/10.1016/j.jclepro.2020.120009

  5. Baykasoğlu A, Madenoğlu FS (2021) Greedy randomized adaptive search procedure for simultaneous scheduling of production and preventive maintenance activities in dynamic flexible job shops. Soft Comput 25(23):14,893–14,932. https://doi.org/10.1007/s00500-021-06053-0

    Article  Google Scholar 

  6. Duan JG, Wang JH (2021) Energy-efficient scheduling for a flexible job shop with machine breakdowns considering machine idle time arrangement and machine speed level selection. Comput Industr Eng 161. https://doi.org/10.1016/j.cie.2021.107677

  7. Gupta S, Jain A (2021) Analysis of integrated preventive maintenance and machine failure in stochastic flexible job shop scheduling with sequence-dependent setup time. Smart Sci:1–23. https://doi.org/10.1080/23080477.2021.1992823

  8. Shahgholi Zadeh M, Katebi Y, Doniavi A (2018) A heuristic model for dynamic flexible job shop scheduling problem considering variable processing times. Int J Prod Res 57(10):3020–3035. https://doi.org/10.1080/00207543.2018.1524165

    Article  Google Scholar 

  9. Li K, Deng Q, Zhang L et al (2021) An effective mcts-based algorithm for minimizing makespan in dynamic flexible job shop scheduling problem. Comput Industr Eng, 155. https://doi.org/10.1016/j.cie.2021.107211

  10. Meng L, Zhang C, Ren Y et al (2020) Mixed-integer linear programming and constraint programming formulations for solving distributed flexible job shop scheduling problem. Comput Industr Eng 142. https://doi.org/10.1016/j.cie.2020.106347

  11. Yan B, Bragin MA, Luh PB (2018) Novel formulation and resolution of job-shop scheduling problems. IEEE Robot Autom Lett 3(4):3387–3393. https://doi.org/10.1109/lra.2018.2850056

    Article  Google Scholar 

  12. Kouider A, Ait HaddadÈNe H (2021) A bi-objective branch-and-bound algorithm for the unit-time job shop scheduling: a mixed graph coloring approach. Comput Oper Res 132. https://doi.org/10.1016/j.cor.2021.105319

  13. Sweeney KD, Sweeney DC, Campbell JF (2019) The performance of priority dispatching rules in a complex job shop: a study on the upper mississippi river. Int J Prod Econ 216:154–172. https://doi.org/10.1016/j.ijpe.2019.04.024

    Article  Google Scholar 

  14. Gao K, Cao Z, Zhang L et al (2019) A review on swarm intelligence and evolutionary algorithms for solving flexible job shop scheduling problems. IEEE/CAA J Automatica Sinica 6(4):904–916. https://doi.org/10.1109/jas.2019.1911540

    Article  Google Scholar 

  15. Mirjalili S, Gandomi AH, Mirjalili SZ et al (2017) Salp swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv Eng Softw 114:163–191. https://doi.org/10.1016/j.advengsoft.2017.07.002

    Article  Google Scholar 

  16. Sun ZX, Hu R, Qian B et al (2018) Salp swarm algorithm based on blocks on critical path for reentrant job shop scheduling problems. In: International conference on intelligent computing. Springer, pp 638–648. https://doi.org/10.1007/978-3-319-95930-6_64

  17. Jouhari H, Lei D, Al-qaness MAA et al (2020) Modified harris hawks optimizer for solving machine scheduling problems. Symmetry 12(9). https://doi.org/10.3390/sym12091460

  18. Ewees AA, Al-qaness MAA, Abd Elaziz M (2021) Enhanced salp swarm algorithm based on firefly algorithm for unrelated parallel machine scheduling with setup times. Appl Math Model 94:285–305. https://doi.org/10.1016/j.apm.2021.01.017

    Article  MathSciNet  MATH  Google Scholar 

  19. Liu C, Yao Y, Zhu H (2021) Hybrid salp swarm algorithm for solving the green scheduling problem in a double-flexible job shop. Appl Sci 12(1). https://doi.org/10.3390/app12010205

  20. Chen F, Gao Y, Chen Zq et al (2005) Scga: controlling genetic algorithms with sarsa (0). In: International conference on computational intelligence for modelling, control and automation and international conference on intelligent agents, web technologies and internet commerce (CIMCA-IAWTIC’06), vol 1. IEEE, pp 1177–1183. https://doi.org/10.1109/cimca.2005.1631422

  21. Hessel M, Modayil J, Van Hasselt H et al (2018) Rainbow: combining improvements in deep reinforcement learning. Proc AAAI conf Artif Intell 32(1). https://doi.org/10.1609/aaai.v32i1.11796

  22. Para J, Del Ser J, Nebro AJ (2022) Energy-aware multi-objective job shop scheduling optimization with metaheuristics in manufacturing industries: a critical survey, results, and perspectives. Appl Sci 12(3). https://doi.org/10.3390/app12031491

  23. Mahmud S, Abbasi A, Chakrabortty RK et al (2021) Multi-operator communication based differential evolution with sequential tabu search approach for job shop scheduling problems. Appl Soft Comput, vol 108. https://doi.org/10.1016/j.asoc.2021.107470

  24. Wu X, Che A (2020) Energy-efficient no-wait permutation flow shop scheduling by adaptive multi-objective variable neighborhood search. Omega 94. https://doi.org/10.1016/j.omega.2019.102117

  25. Abdelmaguid TF (2020) Scatter search with path relinking for multiprocessor open shop scheduling. Comput Industr Eng 141. https://doi.org/10.1016/j.cie.2020.106292

  26. Molina D, Poyatos J, Ser JD et al (2020) Comprehensive taxonomies of nature- and bio-inspired optimization: Inspiration versus algorithmic behavior, critical analysis recommendations. Cognit Computat 12(5):897–939. https://doi.org/10.1007/s12559-020-09730-8

    Article  Google Scholar 

  27. Holland JH (1992) Genetic algorithms. Sci Amer 267(1):66–73

    Article  Google Scholar 

  28. Storn R, Price K (1997) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359. https://doi.org/10.1023/A:1008202821328

    Article  MathSciNet  MATH  Google Scholar 

  29. Simon D (2008) Biogeography-based optimization. IEEE Trans Evolution Computat 12(6):702–713. https://doi.org/10.1109/TEVC.2008.919004

    Article  Google Scholar 

  30. Sulaiman MH, Mustaffa Z, Saari MM et al (2020) Barnacles mating optimizer: a new bio-inspired algorithm for solving engineering optimization problems. Eng Appl Artif Intell 87:103–330. https://doi.org/10.1016/j.engappai.2019.103330

    Article  Google Scholar 

  31. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95 - international conference on neural networks. IEEE. https://doi.org/10.1109/icnn.1995.488968

  32. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39(3):459–471. https://doi.org/10.1007/s10898-007-9149-x

    Article  MathSciNet  MATH  Google Scholar 

  33. Mirjalili S, Mirjalili SM, Lewis A (2014) Grey wolf optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  34. Gharehchopogh BAS, Mirjalili S (2021) Artificial gorilla troops optimizer: a new nature-inspired metaheuristic algorithm for global optimization problems. Int J Intell Syst 36 (10):5887–5958. https://doi.org/10.1002/int.22535

    Article  Google Scholar 

  35. Abdollahzadeh B, Gharehchopogh FS, Mirjalili S (2021) African vultures optimization algorithm: a new nature-inspired metaheuristic algorithm for global optimization problems. Comput Industr Eng 158:107–408. https://doi.org/10.1016/j.cie.2021.107408

    Article  Google Scholar 

  36. Kaur S, Awasthi LK, Sangal A et al (2020) Tunicate swarm algorithm: a new bio-inspired based metaheuristic paradigm for global optimization. Eng Appl Artif Intell 90:103–541. https://doi.org/10.1016/j.engappai.2020.103541

    Article  Google Scholar 

  37. Zhong C, Li G, Meng Z (2022) Beluga whale optimization: a novel nature-inspired metaheuristic algorithm. Knowl-Based Syst 251:109–215. https://doi.org/10.1016/j.knosys.2022.109215

    Article  Google Scholar 

  38. Atashpaz-Gargari E, Lucas C (2007) Imperialist competitive algorithm: an algorithm for optimization inspired by imperialistic competition. In: 2007 IEEE congress on evolutionary computation. IEEE. https://doi.org/10.1109/cec.2007.4425083

  39. Rao R, Savsani V, Vakharia D (2011) Teaching learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43(3):303–315. https://doi.org/10.1016/j.cad.2010.12.015

    Article  Google Scholar 

  40. Jahangiri M, Hadianfard MA, Najafgholipour MA et al (2020) Interactive autodidactic school: a new metaheuristic optimization algorithm for solving mathematical and structural design optimization problems. Comput Structures 235:106–268. https://doi.org/10.1016/j.compstruc.2020.106268

    Article  Google Scholar 

  41. Emami H (2021) Stock exchange trading optimization algorithm: a human-inspired method for global optimization. J Supercomputing 78(2):2125–2174. https://doi.org/10.1007/s11227-021-03943-w

    Article  Google Scholar 

  42. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680. https://doi.org/10.1126/science.220.4598.671

    Article  MathSciNet  MATH  Google Scholar 

  43. Rashedi E, Nezamabadi-Pour H, Saryazdi S (2009) GSA: a gravitational search algorithm. Inf Sci 179(13):2232–2248. https://doi.org/10.1016/j.ins.2009.03.004

    Article  MATH  Google Scholar 

  44. Salimi H (2015) Stochastic fractal search: a powerful metaheuristic algorithm. Knowl-Based Syst 75:1–18. https://doi.org/10.1016/j.knosys.2014.07.025

    Article  Google Scholar 

  45. Mirjalili S (2016) SCA: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133. https://doi.org/10.1016/j.knosys.2015.12.022

    Article  Google Scholar 

  46. Kaveh A, Dadras A (2017) A novel meta-heuristic optimization algorithm: Thermal exchange optimization. Adv Eng Softw 110:69–84. https://doi.org/10.1016/j.advengsoft.2017.03.014

    Article  Google Scholar 

  47. Kaveh A, Akbari H, Hosseini SM (2020) Plasma generation optimization: a new physically-based metaheuristic algorithm for solving constrained optimization problems. Eng Comput 38(4):1554–1606. https://doi.org/10.1108/ec-05-2020-0235

    Article  Google Scholar 

  48. Jiang T, Zhang C (2018) Application of grey wolf optimization for solving combinatorial problems: job shop and flexible job shop scheduling cases. IEEE Access 6:26,231–26,240. https://doi.org/10.1109/access.2018.2833552

    Article  Google Scholar 

  49. Fan B, Yang W, Zhang Z (2018) Solving the two-stage hybrid flow shop scheduling problem based on mutant firefly algorithm. J Ambient Intell Humanized Comput 10(3):979–990. https://doi.org/10.1007/s12652-018-0903-3

    Article  Google Scholar 

  50. Yu H, Gao Y, Wang L et al (2020) A hybrid particle swarm optimization algorithm enhanced with nonlinear inertial weight and gaussian mutation for job shop scheduling problems. Mathematics 8(8). https://doi.org/10.3390/math8081355

  51. Liu M, Yao X, Li Y (2020) Hybrid whale optimization algorithm enhanced with lévy flight and differential evolution for job shop scheduling problems. Appl Soft Comput, vol 87. https://doi.org/10.1016/j.asoc.2019.105954

  52. Li Y, Huang W, Wu R et al (2020) An improved artificial bee colony algorithm for solving multi-objective low-carbon flexible job shop scheduling problem. Appl Soft Comput 95. https://doi.org/10.1016/j.asoc.2020.106544

  53. Kurdi M (2022) Ant colony optimization with a new exploratory heuristic information approach for open shop scheduling problem. Knowl-Based Syst 242. https://doi.org/10.1016/j.knosys.2022.108323

  54. Nadimi-Shahraki MH, Taghian S, Mirjalili S et al (2022) GGWO: gaze cues learning-based grey wolf optimizer and its applications for solving engineering problems. J Computat Sci 61:101–636. https://doi.org/10.1016/j.jocs.2022.101636

    Google Scholar 

  55. Huang C, Li Y, Yao X (2020) A survey of automatic parameter tuning methods for metaheuristics. IEEE Trans Evol Comput 24(2):201–216. https://doi.org/10.1109/tevc.2019.2921598

    Article  Google Scholar 

  56. Shahrabi J, Adibi MA, Mahootchi M (2017) A reinforcement learning approach to parameter estimation in dynamic job shop scheduling. Comput Industr Engi 110:75–82. https://doi.org/10.1016/j.cie.2017.05.026

    Article  Google Scholar 

  57. Emary E, Zawbaa HM, Grosan C (2018) Experienced gray wolf optimization through reinforcement learning and neural networks. IEEE Trans Neural Netw Learn Syst 29(3):681–694. https://doi.org/10.1109/TNNLS.2016.2634548

    Article  MathSciNet  Google Scholar 

  58. Zhao F, Zhang L, Cao J et al (2021) A cooperative water wave optimization algorithm with reinforcement learning for the distributed assembly no-idle flowshop scheduling problem. Comput Industr Eng 153. https://doi.org/10.1016/j.cie.2020.107082

  59. Chen RH, Yang B, Li S et al (2020) A self-learning genetic algorithm based on reinforcement learning for flexible job-shop scheduling problem. Comput Industr Eng 149. https://doi.org/10.1016/j.cie.2020.106778

  60. Wang H, Liu N, Zhang Y et al (2020) Deep reinforcement learning: a survey. Frontiers Inf Technol Electr Eng 21(12):1726–1744. https://doi.org/10.1631/fitee.1900533

    Article  Google Scholar 

  61. Rupprecht T, Wang Y (2022) A survey for deep reinforcement learning in markovian cyber-physical systems: common problems and solutions. Neural Netw 153:13–36. https://doi.org/10.1016/j.neunet.2022.05.013

    Article  Google Scholar 

  62. Xu D, Zhu F, Liu Q et al (2021) Improving exploration efficiency of deep reinforcement learning through samples produced by generative model. Expert Syst Appl 185. https://doi.org/10.1016/j.eswa.2021.115680

  63. Giffler B, Thompson GL (1960) Algorithms for solving production-scheduling problems. Oper Res 8(4):487–503. https://doi.org/10.1287/opre.8.4.487

    Article  MathSciNet  MATH  Google Scholar 

  64. Shi F, Zhao S, Meng Y (2019) Hybrid algorithm based on improved extended shifting bottleneck procedure and ga for assembly job shop scheduling problem. Int J Prod Res 58(9):2604–2625. https://doi.org/10.1080/00207543.2019.1622052

    Article  Google Scholar 

  65. Ge H, Du W, Qian F (2007) A hybrid algorithm based on particle swarm optimization and simulated annealing for job shop scheduling. In: Third international conference on natural computation (ICNC 2007), pp 715–719. https://doi.org/10.1109/ICNC.2007.44

  66. Fisher H (1963) Probabilistic learning combinations of local job-shop scheduling rules. Industr Sched 225–251

  67. Lawrence S (1984) Resouce constrained project scheduling: an experimental investigation of heuristic scheduling techniques (supplement). Graduate school of industrial administration, Carnegie-Mellon University

  68. Mirjalili S (2016) Sca: a sine cosine algorithm for solving optimization problems. Knowl-Based Syst 96:120–133. https://doi.org/10.1016/j.knosys.2015.12.022

    Article  Google Scholar 

  69. Asadzadeh L, Zamanifar K (2010) An agent-based parallel approach for the job shop scheduling problem with genetic algorithms. Math Comput Model 52(11-12):1957–1965. https://doi.org/10.1016/j.mcm.2010.04.019

    Article  MATH  Google Scholar 

  70. Asadzadeh L (2015) A local search genetic algorithm for the job shop scheduling problem with intelligent agents. Comput Industr Eng 85:376–383. https://doi.org/10.1016/j.cie.2015.04.006

    Article  Google Scholar 

  71. Zhao F, Shao Z, Wang J et al (2015) A hybrid differential evolution and estimation of distribution algorithm based on neighbourhood search for job shop scheduling problems. Int J Prod Res 54(4):1039–1060. https://doi.org/10.1080/00207543.2015.1041575

    Article  Google Scholar 

  72. Zhao F, Jiang X, Zhang C et al (2014) A chemotaxis-enhanced bacterial foraging algorithm and its application in job shop scheduling problem. Int J Comput Integr Manuf:1–16. https://doi.org/10.1080/0951192x.2014.961962

Download references

Funding

This research is sponsored by the basic scientific research projects of State Administration of Science, Technology and Industry for National Defense(JCKY2018210A001).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Tongji University, 201804, Shanghai, China

    Yiming Gu, Ming Chen & Liang Wang

Authors
  1. Yiming Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. The first draft of the manuscript was written by Yiming Gu and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Liang Wang.

Ethics declarations

Ethics approval

The authors confirm that they have complied with the publication ethics and state that this work is original and has not been used for publication anywhere before.

Consent for Publication

The authors give consent to the journal regarding the publication of this work.

Competing interests

The authors declare no competing interests.

Additional information

Consent to participate

The authors are willing to participate in journal promotions and updates.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, Y., Chen, M. & Wang, L. A self-learning discrete salp swarm algorithm based on deep reinforcement learning for dynamic job shop scheduling problem. Appl Intell 53, 18925–18958 (2023). https://doi.org/10.1007/s10489-023-04479-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04479-7

Keywords

Search

Navigation