Abstract
Matrix production systems (MPSs) offer significant advantages in flexibility and scalability when compared to conventional line-based production systems. However, they also pose major challenges when it comes to finding optimal decision policies for production planning and control, which is crucial to ensure that flexibility does not come at the cost of productivity. While standard planning methods such as decision rules or metaheuristics suffer from low solution quality and long computation times as problem complexity increases, search methods such as Monte Carlo Tree Search (MCTS) with Reinforcement Learning (RL) have proven powerful in optimising otherwise inhibitively complex problems. Despite its success, open questions remain as to when RL can be beneficial for industrial-scale problems. In this paper, we consider the application of MCTS with RL for optimising the reconfiguration of an MPS. We define two operational scenarios and evaluate the potential of RL in each. Taken more generally, our results provide context to better understand when RL can be beneficial in industrial-scale use cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bortolini, M., Galizia, F.G., Mora, C.: Reconfigurable manufacturing systems: literature review and research trend. J. Manuf. Syst. 49, 93–106 (2018)
Greschke, P., Schönemann, M., Thiede, S., Herrmann, C.: Matrix structures for high volumes and flexibility in production systems. Procedia CIRP 17, 160–165 (2014)
Bortolini, M., Galizia, F.G., Mora, C., Pilati, F.: Reconfigurability in cellular manufacturing systems: a design model and multi-scenario analysis. Int. J. Adv. Manuf. Technol. 104(9), 4387–4397 (2019)
Perwitz, J., Sobottka, T., Beicher, J.N., Gaal, A.: Simulation-based evaluation of performance benefits from flexibility in assembly systems and matrix production. Procedia CIRP 107, 693–698 (2022)
Joseph, O.A., Sridharan, R.: Effects of routing flexibility, sequencing flexibility and scheduling decision rules on the performance of a flexible manufacturing system. Int. J. Adv. Manuf. Technol. 56(1), 291–306 (2011)
Zhu, Q., Huang, S., Wang, G., Moghaddam, S.K., Lu, Y., Yan, Y.: Dynamic reconfiguration optimization of intelligent manufacturing system with human-robot collaboration based on digital twin. J. Manuf. Syst. 65, 330–338 (2022)
Luo, K., Shen, G., Li, L., Sun, J.: 0–1 mathematical programming models for flexible process planning. Eur. J. Oper. Res. 308(3), 1160–1175 (2023)
Rodrigues, N., Oliveira, E., Leitão, P.: Decentralized and on-the-fly agent-based service reconfiguration in manufacturing systems. Comput. Ind. 101, 81–90 (2018)
Mo, F., et al.: A framework for manufacturing system reconfiguration and optimisation utilising digital twins and modular artificial intelligence. Robot. Comput.-Integr. Manuf. 82, 102524 (2023)
Morariu, C., Morariu, O., Răileanu, S., Borangiu, T.: Machine learning for predictive scheduling and resource allocation in large scale manufacturing systems. Comput. Ind. 120, 103244 (2020)
Scrimieri, D., Adalat, O., Afazov, S., Ratchev, S.: Modular reconfiguration of flexible production systems using machine learning and performance estimates. IFAC-PapersOnLine 55(10), 353–358 (2022)
Yang, S., Xu, Z.: Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing. Int. J. Prod. Res. 60(16), 4936–4953 (2022)
Monka, P.P., Monkova, K., Jahnátek, A., Vanca, J.: Flexible manufacturing system simulation and optimization. In: Mitrovic, N., Mladenovic, G., Mitrovic, A. (eds.) CNNTech 2020. LNNS, vol. 153, pp. 53–64. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-58362-0_4
Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Silver, D., et al.: A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362(6419), 1140–1144 (2018)
Fawzi, A., et al.: Discovering faster matrix multiplication algorithms with reinforcement learning. Nature 610(7930), 47–53 (2022)
Halbwidl, C., Sobottka, T., Gaal, A., Sihn, W.: Deep reinforcement learning as an optimization method for the configuration of adaptable, cell-oriented assembly systems. Procedia CIRP 104, 1221–1226 (2021)
Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 282–293. Springer, Heidelberg (2006). https://doi.org/10.1007/11871842_29
Göppert, A., Mohring, L., Schmitt, R.H.: Predicting performance indicators with ANNs for AI-based online scheduling in dynamically interconnected assembly systems. Prod. Eng. Res. Devel. 15(5), 619–633 (2021)
Cobbe, K., Klimov, O., Hesse, C., Kim, T., Schulman, J.: Quantifying Generalization in Reinforcement Learning, July 2019
Kirk, R., Zhang, A., Grefenstette, E., Rocktäschel, T.: A survey of zero-shot generalisation in deep reinforcement learning. J. Artif. Intell. Res. 76, 201–264 (2023)
Acknowledgements
This research has been supported by the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology (BMK), the German Federal Ministry for Economic Affairs and Climate Action (BMWK) and the Fraunhofer-Gesellschaft through the projects REINFORCE (887500), champI4.0ns (891793) and MES.Trix.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Czarnetzki, L., Laflamme, C., Halbwidl, C., Günther, L.C., Sobottka, T., Bachlechner, D. (2023). Optimisation of Matrix Production System Reconfiguration with Reinforcement Learning. In: Seipel, D., Steen, A. (eds) KI 2023: Advances in Artificial Intelligence. KI 2023. Lecture Notes in Computer Science(), vol 14236. Springer, Cham. https://doi.org/10.1007/978-3-031-42608-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-42608-7_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42607-0
Online ISBN: 978-3-031-42608-7
eBook Packages: Computer ScienceComputer Science (R0)