Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning
- 646 Downloads
In recent years, most companies have resorted to multi-site or supply-chain organization in order to improve their competitiveness and adapt to existing real conditions. In this article, a model for adaptive scheduling in multi-site companies is proposed. To do this, a multi-agent approach is adopted in which intelligent agents have reactive learning capabilities based on reinforcement learning. This reactive learning technique allows the agents to make accurate short-term decisions and to adapt these decisions to environmental fluctuations. The proposed model is implemented on a 3-tier architecture that ensures the security of the data exchanged between the various company sites. The proposed approach is compared to a genetic algorithm and a mixed integer linear program algorithm to prove its feasibility and especially, its reactivity. Experimentations on a real case study demonstrate the applicability and the effectiveness of the model in terms of both optimality and reactivity.
KeywordsProduction control Scheduling Multi-agent system Reinforcement learning Multi-site company
Unable to display preview. Download preview PDF.
- Aissani, N., Beldjilali, B., & Trentesaux, D. (2008b). Efficient and effective reactive scheduling of manufacturing system using SARSA-multi-objective-agents. In Proceedings of the 7th international conference MOSIM, Paris, pp. 698–707.Google Scholar
- Ait Si Larbi, E. Y., Aissani, N., & Beldjilali, B. (2008). Un Modèle de Planification pour les Entreprises Multi Sites basé sur les Systèmes Multi Agents et les Algorithmes Génétiques. In Proceedings of the 10th maghrebian conference on information technologies, Oran, Algeria, pp. 506–511.Google Scholar
- Conway R. W., Maxwell W. L., Miller L. W. (1967) Theory of scheduling. Addison-Wesley, Reading, MAGoogle Scholar
- Dabbene, F., Gay, P., Tortia, C., & Sacco, N. (2005). Optimization of fresh–food supply chains in uncertain environments: An application to the meat-refrigeration process, decision and control, European Control Conference. CDC-ECC;05. 44th IEEE Conference on Volume 12, Issue 15, pp. 2077–2082.Google Scholar
- Dauzere-Peres S., & Paulli, J. (1994). Solving the general job-shop scheduling problem. Management. Report Series, vol. 182. Erasmus University Rotterdam, Rotterdam School of Management, Rotterdam.Google Scholar
- Ferber, J., & Gutknecht, O. (1998). A meta-model for the analysis and design of organizations in multi-agent systems. In Proceedings ICMAS’98, pp. 128–135.Google Scholar
- Fontan, G., Merce, C., & Erschler, J. (2001). La planification des flux de production, Performance industrielle et gestion des flux, Hermes Lavoisier, Traité IC2 Information-Commande-Communication, 2001, Chap 3, pp. 69–112.Google Scholar
- Galliano, D., & Soulie, N. (2007). Organisational and spatial determinants of the multiunit firm: Evidence from the French indus, Cahiers du GRES, No 17.Google Scholar
- Goldberg, D. E., & Lingle, R. (1985). Alleles, loci and the traveling salesman problem. In Proceedings of the first international conference on Genetic Algorithms, pp. 10–19.Google Scholar
- Haruno, M., & Kawato, M. (2006). Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: FMRI examination in stimulus-action-reward association learning. Neural Networks, 19(Special Issue), 1242–1254.Google Scholar
- Holland J. H. (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, ILGoogle Scholar
- Hong, G., Chuan, L., Zhang, L., & Xianming, Z. (2007) Study on supply chain optimization scheduling of networked manufacturing, ACOS’07: In Proceedings of WSEAS: 6th international conference on applied computer science, Vol 6.Google Scholar
- IBM. (2010). IBM ILOG CPLEX optimizer, high performance mathematical optimization engines. http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/.
- James, T. L., Brown, E. C., & Keeling, K. B. (2007). A hybrid grouping genetic algorithm for the cell formation problem. Computers and operations research. No 34, pp. 2059–2079.Google Scholar
- Katalinic, B., & Kordic, V. (2004). Bionic assembly system: Concept, structure and function. In Proceedings of the 5th IDMME, Bath, UK.Google Scholar
- Lee H.L., Rosenblatt J. (1986) A generalized quantity discount pricing model to increase supplier’s profits. Management Science 33(9): 1167–1185Google Scholar
- Marquès, G., Lamothe, J., Thierry, C., & Gourc, D. (2009). A supply chain performance analysis of a pull inspired supply strategy faced to demande uncertainties. Journal of Intel Manufacturing, 20(6). doi: 10.1007/s10845-009-0337-z.
- Mati, Y., Lahlou, C., & Dauzère-Pérès, S.(2010). Modelling and solving a practical flexible job-shop scheduling problem with blocking constraints. International Journal of Production Research, 1366-588X, First published on 23 Sep. 2010.Google Scholar
- McCulloch W. S. (1945) A heterarchy of values determined by the topology of nervous nets. Bulletin of Mathematical Biology 7: 89–93Google Scholar
- Miles, R. E., & Snow, C. C. (1992). Managing 21st century network organisations. Organizational dynamics, winter session.Google Scholar
- Mintzberg, H. (1980). Structure in 5’s: A synthesis of the research. Organization Design Management Science, 26(3).Google Scholar
- Mourtzis, D., Papakostas, N., Makris, S., Xanthakis, V., & Chryssolouris, G. (2008). Supply chain modeling and control for producing highly customized products. General assembly of CIRP No58, Manchester, UK (24/08/2008), Vol. 57, No 1, p. 588.Google Scholar
- Ounnar, F., & Pujo, P. (2009). Pull control for job shop: Holonic manufacturing system approach using multicriteria decision-making. Journal of Intelligent Manufacturing. doi: 10.1007/s10845-009-0288-4.
- Pezzella, F., Morganti, G., & Ciaschetti, G. (2008). A genetic algorithm for the flexible job-shop scheduling problem. Computers & Operations Research, 35(10), Oct. 2008.Google Scholar
- Rummery, G., & Niranjan, M. (1994). On-line q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166. Cambridge University, Engineering Department.Google Scholar
- Russell, S., & Norvig, P. (1995). Artificial intelligence: A modern approach, the intelligent agent book. Prentice Hall Series in Artificial Intelligence.Google Scholar
- Sauer, J., Freese, T., & Teschke, T. (2000). Towards agent-based multi-site scheduling. ECAI 2000 workshop on new results in planning, scheduling and design, pp. 123–130.Google Scholar
- Silva, N., Sousa, P., & Ramos, C. (1998). A holonic manufacturing system implementation. Advanced Summer Institute(ASI’98). Bremen, Germany; 14–17 June 1998.Google Scholar
- Swaminathan, J., Smith, S., & Sadeh-Koniecpol, N. (1997). Modeling supply chain dynamics: A multiagent approach. Decision Sciences.Google Scholar
- Takadama, K., & Fujita, H. (2004). Lessons learned from comparison between Q-learning and sarsa agents in bargaining game. In North American association for computational social and organizational science (NAACSOS 2004), June 27–29, Pittsburgh, PA.Google Scholar
- Thierry, C. (2003). gestion de chaînes logistiques, modèles et mise en œuvre pour l’aide à la décision à moyen terme. HDR thesis, Toulouse 2 university.Google Scholar
- Vilcot, G. (2007). Algorithmes approchés pour des problèmes d’ordonnancement multicritère de type job shop flexible et job shop multiressource, PhD thesis, University of François-Rabelais, Tours, France.Google Scholar
- Voss S., Woodruff D. L. (2006) Introduction to computational optimization models for production planning in supply chain. Springer, BerlinGoogle Scholar
- Watkins, C. J. C. H. (1989). Learning from delayed rewards, PhD thesis, Cambridge University, Cambridge, England.Google Scholar
- Zambrano, G., Aissani, N., Pach, C., Berger, T., & Trentesaux, D. (2011). An approach for temporal myopia reduction in heterarchical control architectures. In Proceedings of 20th IEEE inter symposium on industrial electronics, 27–30 June 2011.Google Scholar
- Zbib, N., Pach, C., Sallez, Y., & Trentesaux, D. (2010). Heterarchical production control in manufacturing systems using the potential fields concept. Journal of Intelligent Manufacturing. doi: 10.1007/s10845-010-0467-3.
- Zhang H., Gen M. (2005) Multistage-based genetic algorithm for flexible job-shop scheduling problem. Journal of Complexity International 11: 223–232Google Scholar
- Zobolas G. I., Tarantilis C. D., Ioannou G. (2008) Exact, Heuristic and Meta-heuristic Algorithms for Solving Shop Scheduling Problems, Studies in Computational Intelligence (SCI). Springer, Berlin, pp 1–40Google Scholar