Skip to main content
SpringerLink
Log in

Distributed Flexible Job Shop Scheduling through Deploying Fog and Edge Computing in Smart Factories Using Dual Deep Q Networks

  • Research
  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Flexible job shop scheduling (FJSP) has garnered enormous attention within the realm of smart manufacturing, where, beyond job sequencing, the selection of machines holds considerable importance. As smart factories progress with the Internet of things (IoT) and cyber-physical systems (CPS), scheduling methodologies are advancing towards intelligent decentralization. However, with the expansion of factories, conventional cloud computing struggles to manage the substantial influx of data. To tackle this issue, this work incorporates a fog computing and edge computing framework into the distributed FJSP workstations. In this framework, the workstations each of which consists of multiple machines are categorized based on the different nature of the accommodated machines, and operate independently to reduce unnecessary information transmission, in which each machine is equipped with edge computing capacity. The fusion of fog computing and edge computing allows for the offloading of computational tasks from cloud computing, effectively reducing latency. While previous solutions for FJSP have predominantly relied on linear programming or metaheuristic algorithms, this work proposed a novel distributed approach based on a dual deep Q networks (dual DQN) architecture, integrating deep learning (DL) with reinforcement learning (RL). Within the cloud center, the initial neural network determines the machine selection rules for fog computing, while the secondary neural network decides the job dispatching rules for edge computing devices. Edge computing devices execute the schedule and provide feedback to the cloud, which refines the results through an iterative training process, so that to minimize the makespan. The experimental findings indicate that employing dual DQNs outperforms the methods of utilizing only one single machine selection rule.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Algorithm 7
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. Garey MR, Johnson DS, Sethi R (1976) The complexity of flowshop and jobshop scheduling. Math Oper Res 1(2):117–129

    Article  MathSciNet  Google Scholar 

  2. Vital-Soto A, Azab A, Baki MF (2020) Mathematical modeling and a hybridized bacterial foraging optimization algorithm for the flexible job-shop scheduling problem with sequencing flexibility. J Manuf Syst 54:74–93

    Article  Google Scholar 

  3. Brucker P, Schlie R (1990) Job-shop scheduling with multipurpose machines. Computing 45:369–375

    Article  MathSciNet  Google Scholar 

  4. Sawik T (1990) Modelling and scheduling of a flexible manufacturing system. Eur J Oper Res 45(2–3):177–190

    Article  Google Scholar 

  5. Gran SS, Ismail I, Ajol TA, Ibrahim AFA (2015) Mixed Integer programming model for flexible job-shop scheduling problem (FJSP) to minimize makespan and total machining time. In: Proc. of 2015 International Conference on Computer, Communications, and Control Technology (I4CT). IEEE Press, pp. 413–417

  6. Ho NB, Tay JC (2005) Evolving dispatching rules for solving the flexible job-shop problem. In: Proc. of 2005 IEEE Congress on Evolutionary Computation. IEEE Press, vol. 3, pp. 2848–2855

  7. Rani S, Ahmed SH, Rastogi R (2020) Dynamic clustering approach based on wireless sensor networks genetic algorithm for IoT applications. Wireless Netw 26:2307–2316

    Article  Google Scholar 

  8. Deepalakshmi P, Shankar K (2020) Role and impacts of ant colony optimization in job shop scheduling problems: A detailed analysis. In: Gandomi AH et al (eds) Evolutionary Computation in Scheduling. pp 11–35

    Chapter  Google Scholar 

  9. Ge HW, Sun L, Liang YC, Qian F (2008) An effective PSO and AIS-based hybrid intelligent algorithm for job-shop scheduling. IEEE Trans Syst Man Cybern Part A Syst Humans 38(2):358–368

    Article  Google Scholar 

  10. Hajibabaei M, Behnamian J (2021) Flexible job-shop scheduling problem with unrelated parallel machines and resources-dependent processing times: a tabu search algorithm. Int J Manag Sci Eng Manag 16(4):242–253

    Google Scholar 

  11. Fan X, Feng X, Dong Y, Hou H (2022) COVID-19 CT image recognition algorithm based on transformer and CNN. Displays 72:102150

    Article  Google Scholar 

  12. Yang MQ, Lin JCW (2022) Digital image feature recognition method of mobile platform based on machine learning. Mobile Netw Appl 27(6):2506–2514

    Article  MathSciNet  Google Scholar 

  13. Padmanabhan J, Johnson Premkumar MJ (2015) Machine learning in automatic speech recognition: A survey. IETE Tech Rev 32(4):240–251

    Article  Google Scholar 

  14. Tian H, Zhu Z, Jing X (2023). Deep learning for depression recognition from speech. Mob Netw Appl. in press

  15. Zang J, You P (2023) An industrial IoT-enabled smart healthcare system using big data mining and machine learning. Wireless Netw 29(2):909–918

    Article  Google Scholar 

  16. Marmolejo-Saucedo JA, Kose U (2022) Numerical grad-CAM based explainable convolutional neural network for brain tumor diagnosis. Mob Netw Appl. in press

  17. Williem RS, Setiawan K (2011) Reinforcement learning combined with radial basis function neural network to solve job-shop scheduling problem. In: Proc. of 2011 IEEE International Summer Conference of Asia Pacific Business Innovation and Technology Management. IEEE Press, pp 29–32

    Google Scholar 

  18. Bouazza W, Sallez Y, Beldjilali B (2017) A distributed approach solving partially flexible job-shop scheduling problem with a Q-learning effect. IFAC-Pap Line 50(1):15890–15895

    Article  Google Scholar 

  19. Cisco estimation report. Accessed from: https://www.cisco.com/c/en/us/solutions/collateral/service-provider/global-cloud-index-gci/white-paper-c11-738085.html#_Toc503317525

  20. Lin CC, Tsai CT, Liu YL, Chang TT, Chang YS (2023) Security and privacy in 5G-IIoT smart factories: Novel approaches, trends, and challenges. Mob Netw Appl. in press

  21. Jia G, Han G, Rao H, Shu L (2017) Edge computing-based intelligent manhole cover management system for smart cities. IEEE Internet Things J 5(3):1648–1656

    Article  Google Scholar 

  22. Yi S, Hao Z, Zhang Q, Zhang Q, Shi W, Li Q (2017) Lavea: Latency-aware video analytics on edge computing platform. In: Proc. of the 2nd ACM/IEEE Symposium on Edge Computing. pp 1–13

    Google Scholar 

  23. Shi W, Cao J, Zhang Q, Li Y, Xu L (2016) Edge computing: vision and challenges. IEEE Internet Things J 3(5):637–646

    Article  Google Scholar 

  24. Carlier J (1982) The problem of scheduling with earliest start and due dates: Two-Machine Algorithms. J Oper Res Soc 33(3):275–282

    Google Scholar 

  25. Stecke KE (1983) Formulation and solution of nonlinear integer production planning problems for flexible manufacturing systems. Manage Sci 29(3):273–288

    Article  Google Scholar 

  26. Berrada M, Stecke KE (1986) A branch and bound approach for machine load balancing in flexible manufacturing systems. Manage Sci 32(10):1316–1335

    Article  Google Scholar 

  27. Calleja G, Pastor R (2014) A dispatching algorithm for flexible job-shop scheduling with transfer batches: an industrial application. Prod Plan Control 25(2):93–109

    Article  Google Scholar 

  28. Liangxiao J, Zhongjun D (2015) An improved genetic algorithm for flexible job shop scheduling problem. In: 2015 2nd international conference on information science and control engineering. IEEE Press. pp. 127–131

  29. Rossi A (2014) Flexible job shop scheduling with sequence-dependent setup and transportation times by ant colony with reinforced pheromone relationships. Int J Prod Econ 153:253–267

    Article  Google Scholar 

  30. Riedmiller S, Riedmiller M (1999) A neural reinforcement learning approach to learn local dispatching policies in production scheduling. In: Proc of IJCAI 1999. vol. 2, pp. 764–771

  31. Tawfeek MA, El-Sisi A, Keshk AE, Torkey FA (2013) Cloud task scheduling based on ant colony optimization. In: Proc. of 2013 8th International Conference on Computer Engineering & Systems (ICCES). IEEE Press, pp 64–69

    Google Scholar 

  32. Zhao S, Ding G, Gao Y, Han J (2017) Approximating discrete probability distribution of image emotions by multi-modal features fusion. Transfer 1000(1):4669–4675

    Google Scholar 

  33. Yin L, Luo J, Luo H (2018) Tasks scheduling and resource allocation in fog computing based on containers for smart manufacturing. IEEE Trans Industr Inf 14(10):4712–4721

    Article  Google Scholar 

  34. Park J, Chun J, Kim SH, Kim Y, Park J (2021) Learning to schedule job-shop problems: representation and policy learning using graph neural network and reinforcement learning. Int J Prod Res 59(11):3360–3377

    Article  Google Scholar 

  35. Zhang L (2009) H∞ estimation for discrete-time piecewise homogeneous Markov jump linear systems. Automatica 45(11):2570–2576

    Article  MathSciNet  Google Scholar 

  36. Fang Y, Peng C, Lou P, Zhou Z, Hu J, Yan J (2019) Digital-twin-based job shop scheduling toward smart manufacturing. IEEE Trans Industr Inf 15(12):6425–6435

    Article  Google Scholar 

  37. Li R, Gong W, Lu C (2022) Self-adaptive multi-objective evolutionary algorithm for flexible job shop scheduling with fuzzy processing time. Comput Ind Eng 168:108099

    Article  Google Scholar 

  38. Bo LI, Chen GUO, Tao NING (2018) An improved bacterial foraging optimization for multi-objective flexible job-shop scheduling problem. J Eur Syst Autom 51(4–6):323

    Google Scholar 

  39. Wang L et al (2021) Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning. Comput Netw 190:107969

    Article  Google Scholar 

  40. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

  41. Sutton RS (1999) Reinforcement learning: past, present and future. In: Proc. of 2nd Asia-Pacific Conference on Simulated Evolution and Learning (SEAL’98). pp 195–197

    Google Scholar 

  42. Hinton GE (2009) Deep belief networks. Scholarpedia 4(5):5947

    Article  Google Scholar 

  43. Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    Article  Google Scholar 

  44. Zhang Y, Brown D (2014) Simulation optimization of police patrol districting plans using response surfaces. Simulation 90(6):687–705

    Article  Google Scholar 

  45. Nouri HE, Belkahla Driss O, Ghédira K (2018) Solving the flexible job shop problem by hybrid metaheuristics-based multiagent model. J Indust Eng Int 14:1–14

    Article  Google Scholar 

  46. Lin CC, Deng DJ, Chih YL, Chiu HT (2019) Smart manufacturing scheduling with edge computing using multiclass deep Q network. IEEE Trans Industr Inf 15(7):4276–4284

    Article  Google Scholar 

  47. Li X, Gao L (2016) An effective hybrid genetic algorithm and tabu search for flexible job shop scheduling problem. Int J Prod Econ 174:93–110

    Article  Google Scholar 

  48. Borreguero-Sanchidrián T, Pulido R, García-Sánchez Á, Ortega-Mier M (2017) Flexible job shop scheduling with operators in aeronautical manufacturing: A case study. IEEE Access 6:224–233

    Article  Google Scholar 

Download references

Funding

This work was sponsored by National Science and Technology Council, Taiwan under Grants NSTC 112-2221-E-A49-116-MY3 and NSTC 112-2221-E-240-001.

Author information

Authors and Affiliations

  1. Department of Industrial Engineering and Management, National Yang Ming Chiao Tung University, Hsinchu, 300, Taiwan

    Chun-Cheng Lin, Yi-Chun Peng, Zhen-Yin Annie Chen & Yu-Hong Fan

  2. International Master Program of Smart Manufacturing and Applied Information Science, National Chin-Yi University of Technology, Taichung, 411, Taiwan

    Chun-Cheng Lin

  3. Department of Information Technology, Overseas Chinese University, Taichung, 407, Taiwan

    Hui-Hsin Chin

Authors
  1. Chun-Cheng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Chun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen-Yin Annie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu-Hong Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hui-Hsin Chin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Chun-Cheng Lin and Hui-Hsin Chin conceived of the presented idea, developed the theory, and verified the analytical methods. Yi-Chun Peng, Zhen-Yin Annie Chen, and Yu-Hong Fan contributed to the design and implementation of the research, and to the writing of the manuscript. All authors discussed the results and contributed to the final manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Hui-Hsin Chin.

Ethics declarations

Ethics Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, CC., Peng, YC., Chen, ZY.A. et al. Distributed Flexible Job Shop Scheduling through Deploying Fog and Edge Computing in Smart Factories Using Dual Deep Q Networks. Mobile Netw Appl (2024). https://doi.org/10.1007/s11036-024-02302-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11036-024-02302-2

Keywords

Search

Navigation