Skip to main content

Advertisement

SpringerLink
Log in

Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In recent years, most companies have resorted to multi-site or supply-chain organization in order to improve their competitiveness and adapt to existing real conditions. In this article, a model for adaptive scheduling in multi-site companies is proposed. To do this, a multi-agent approach is adopted in which intelligent agents have reactive learning capabilities based on reinforcement learning. This reactive learning technique allows the agents to make accurate short-term decisions and to adapt these decisions to environmental fluctuations. The proposed model is implemented on a 3-tier architecture that ensures the security of the data exchanged between the various company sites. The proposed approach is compared to a genetic algorithm and a mixed integer linear program algorithm to prove its feasibility and especially, its reactivity. Experimentations on a real case study demonstrate the applicability and the effectiveness of the model in terms of both optimality and reactivity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aissani N., Trentesaux D., Beldjilali B. (2008) Use of machine learning for continuous improvement of the real time manufacturing control system performances. International Journal of Industrial System Engineering 3(4): 474–497

    Article  Google Scholar 

  • Aissani, N., Beldjilali, B., & Trentesaux, D. (2008b). Efficient and effective reactive scheduling of manufacturing system using SARSA-multi-objective-agents. In Proceedings of the 7th international conference MOSIM, Paris, pp. 698–707.

  • Aissani N., Trentesaux D., Beldjilali B. (2009) Dynamic scheduling of maintenance tasks in the petroleum industry: A reinforcement approach. EAAI: Engineering Applications of Artificial Intelligence 22: 1089–1103

    Article  Google Scholar 

  • Ait Si Larbi, E. Y., Aissani, N., & Beldjilali, B. (2008). Un Modèle de Planification pour les Entreprises Multi Sites basé sur les Systèmes Multi Agents et les Algorithmes Génétiques. In Proceedings of the 10th maghrebian conference on information technologies, Oran, Algeria, pp. 506–511.

  • Bousbia S., Trentesaux D. (2002) Self-organization in distributed manufacturing control: State-of-the-art and future trends. IEEE International Conference on Systems, Man & Cybernetics 5: 6

    Article  Google Scholar 

  • Brucker P., Schlie R. (1990) Job shop scheduling with multi-purpose machines. Computing 45: 369–375

    Article  Google Scholar 

  • Brandimarte P. (1993) Routing and scheduling in a flexible job shop by tabu search. Annals of Operations Research 41: 157–183

    Article  Google Scholar 

  • Chuin Lau H., Agussurja L., Thangarajoo R. (2008) Real-time supply chain control via multi-agent adjustable autonomy. Computers and Operations Research 35(11): 3452–3464

    Article  Google Scholar 

  • Conway R. W., Maxwell W. L., Miller L. W. (1967) Theory of scheduling. Addison-Wesley, Reading, MA

    Google Scholar 

  • Dabbene, F., Gay, P., Tortia, C., & Sacco, N. (2005). Optimization of fresh–food supply chains in uncertain environments: An application to the meat-refrigeration process, decision and control, European Control Conference. CDC-ECC;05. 44th IEEE Conference on Volume 12, Issue 15, pp. 2077–2082.

  • Dauzere-Peres S., & Paulli, J. (1994). Solving the general job-shop scheduling problem. Management. Report Series, vol. 182. Erasmus University Rotterdam, Rotterdam School of Management, Rotterdam.

  • Dauzère-Pérès S., Paulli J. (1997) An integrated approach for modeling and solving the general multiprocessor job-shop scheduling problem using tabu search. Annals of Operations Resarch 70: 281–306

    Article  Google Scholar 

  • Dongbing G., Yang E. (2007) Fuzzy policy reinforcement learning in cooperative multi-robot systems. Journal of Intelligent and Robotic Systems 48(1): 7–22

    Article  Google Scholar 

  • Fattahi P., Saidi Mehrabad M., Jolai F. (2007) Mathematical modelling and heuristic approaches to flexible job shop scheduling problems. Journal of Intelligent and Manufacturing 18: 331–342

    Article  Google Scholar 

  • Ferber, J., & Gutknecht, O. (1998). A meta-model for the analysis and design of organizations in multi-agent systems. In Proceedings ICMAS’98, pp. 128–135.

  • Fontan, G., Merce, C., & Erschler, J. (2001). La planification des flux de production, Performance industrielle et gestion des flux, Hermes Lavoisier, Traité IC2 Information-Commande-Communication, 2001, Chap 3, pp. 69–112.

  • Galliano, D., & Soulie, N. (2007). Organisational and spatial determinants of the multiunit firm: Evidence from the French indus, Cahiers du GRES, No 17.

  • Gao J., Gen M., Sun L., Zhao X. (2007) A hybrid of genetic algorithm and bottleneck shifting for multiobjective flexible job shop scheduling problems. Computers and Industrial Engineering 53: 149–162

    Article  Google Scholar 

  • Goldberg, D. E., & Lingle, R. (1985). Alleles, loci and the traveling salesman problem. In Proceedings of the first international conference on Genetic Algorithms, pp. 10–19.

  • Haruno, M., & Kawato, M. (2006). Heterarchical reinforcement-learning model for integration of multiple cortico-striatal loops: FMRI examination in stimulus-action-reward association learning. Neural Networks, 19(Special Issue), 1242–1254.

    Google Scholar 

  • Holland J. H. (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, IL

    Google Scholar 

  • Hong, G., Chuan, L., Zhang, L., & Xianming, Z. (2007) Study on supply chain optimization scheduling of networked manufacturing, ACOS’07: In Proceedings of WSEAS: 6th international conference on applied computer science, Vol 6.

  • IBM. (2010). IBM ILOG CPLEX optimizer, high performance mathematical optimization engines. http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/.

  • James, T. L., Brown, E. C., & Keeling, K. B. (2007). A hybrid grouping genetic algorithm for the cell formation problem. Computers and operations research. No 34, pp. 2059–2079.

    Google Scholar 

  • Kacem I., Hammadi S., Borne P. (2002) Approach by localization and multiobjective evolutionary optimization for flexible job shop scheduling problems. IEEE Transactions on Systems, Man and Cybernetics, Part C 32(1): 408–419

    Article  Google Scholar 

  • Katalinic, B., & Kordic, V. (2004). Bionic assembly system: Concept, structure and function. In Proceedings of the 5th IDMME, Bath, UK.

  • Lee H.L., Rosenblatt J. (1986) A generalized quantity discount pricing model to increase supplier’s profits. Management Science 33(9): 1167–1185

    Google Scholar 

  • Leitao P., Restivo F. (2008) A holonic approach to dynamic manufacturing scheduling. Robotics and Computer-Integrated Manufacturing 24: 625–634

    Article  Google Scholar 

  • Marquès, G., Lamothe, J., Thierry, C., & Gourc, D. (2009). A supply chain performance analysis of a pull inspired supply strategy faced to demande uncertainties. Journal of Intel Manufacturing, 20(6). doi:10.1007/s10845-009-0337-z.

  • Mastrolilli M., Gambardella L. M. (2000) Effective neighbourhood functions for the flexible job shop problem. Journal of Scheduling 3: 3–20

    Article  Google Scholar 

  • Mati Y., Rezg N., Xie X.L. (2001) Geometric approach and taboo search for scheduling flexible manufacturing systems. IEEE Transactions on Robotics and Automation 17: 805–818

    Article  Google Scholar 

  • Mati, Y., Lahlou, C., & Dauzère-Pérès, S.(2010). Modelling and solving a practical flexible job-shop scheduling problem with blocking constraints. International Journal of Production Research, 1366-588X, First published on 23 Sep. 2010.

  • McCulloch W. S. (1945) A heterarchy of values determined by the topology of nervous nets. Bulletin of Mathematical Biology 7: 89–93

    Google Scholar 

  • Miles, R. E., & Snow, C. C. (1992). Managing 21st century network organisations. Organizational dynamics, winter session.

  • Mintzberg, H. (1980). Structure in 5’s: A synthesis of the research. Organization Design Management Science, 26(3).

  • Monahan J. P. (1984) A quantity pricing model to increase vendor profits. Management Science 30(6): 720–726

    Article  Google Scholar 

  • Monostori L., Csáji B. Cs., Kádár B. (2004) Adaptation and learning in distributed production control. CIRP Annals-Manufacturing Technology 53(1): 349–352

    Article  Google Scholar 

  • Mourtzis, D., Papakostas, N., Makris, S., Xanthakis, V., & Chryssolouris, G. (2008). Supply chain modeling and control for producing highly customized products. General assembly of CIRP No58, Manchester, UK (24/08/2008), Vol. 57, No 1, p. 588.

  • Narasimhan R., Mahapatra S. (2004) Decision models in global supply chain management. Industrial Marketing Management 33: 21–27

    Article  Google Scholar 

  • Ouhimmou M., D’Amours S., Beauregard R., Ait-Kadi D., Singh Chauhan S. (2008) Furniture supply chain tactical planning optimization using a time decomposition approach. European Journal of Operational Research 189(3): 952–970

    Article  Google Scholar 

  • Ounnar, F., & Pujo, P. (2009). Pull control for job shop: Holonic manufacturing system approach using multicriteria decision-making. Journal of Intelligent Manufacturing. doi:10.1007/s10845-009-0288-4.

  • Pezzella, F., Morganti, G., & Ciaschetti, G. (2008). A genetic algorithm for the flexible job-shop scheduling problem. Computers & Operations Research, 35(10), Oct. 2008.

  • Prabhu V. V. (2003) Stability and fault adaptation in distributed control of heterarchical manufacturing job shops. IEEE Transactions on Robotics and Automation 19(1): 142–147

    Article  Google Scholar 

  • Rummery, G., & Niranjan, M. (1994). On-line q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166. Cambridge University, Engineering Department.

  • Russell, S., & Norvig, P. (1995). Artificial intelligence: A modern approach, the intelligent agent book. Prentice Hall Series in Artificial Intelligence.

  • Sallez Y., Berger T., Trentesaux D. (2009) A stigmergic approach for dynamic routing of active products in FMS. Computers in Industry 60(3): 204–216

    Article  Google Scholar 

  • Sauer, J., Freese, T., & Teschke, T. (2000). Towards agent-based multi-site scheduling. ECAI 2000 workshop on new results in planning, scheduling and design, pp. 123–130.

  • Silva, N., Sousa, P., & Ramos, C. (1998). A holonic manufacturing system implementation. Advanced Summer Institute(ASI’98). Bremen, Germany; 14–17 June 1998.

  • Swaminathan, J., Smith, S., & Sadeh-Koniecpol, N. (1997). Modeling supply chain dynamics: A multiagent approach. Decision Sciences.

  • Takadama, K., & Fujita, H. (2004). Lessons learned from comparison between Q-learning and sarsa agents in bargaining game. In North American association for computational social and organizational science (NAACSOS 2004), June 27–29, Pittsburgh, PA.

  • Tang C. S. (1990) The impact of uncertainty on a production line. Management Science 36(12): 1518–1531

    Article  Google Scholar 

  • Tarantilis C. D. (2008) Topics in real-time supply chain management. Computers & Operations Research 35(11): 3393–3396

    Article  Google Scholar 

  • TehraniNik Nejad H., Sugimura N., Iwamura K., Tanimizu Y. (2008) Multi agent architecture for dynamic incremental process planning in the flexible manufacturing system. Journal of Intelligent Manufacturing 21(4): 487–499

    Article  Google Scholar 

  • Thierry, C. (2003). gestion de chaînes logistiques, modèles et mise en œuvre pour l’aide à la décision à moyen terme. HDR thesis, Toulouse 2 university.

  • Trentesaux D., Pesin P., Tahon C. (2000) Distributed artificial intelligence for FMS scheduling, control and design support. Journal of Intelligent Manufacturing 11: 573–589

    Article  Google Scholar 

  • Trentesaux D. (2009) Distributed control of production systems. Engineering Applications of Artificial Intelligence 22(7): 971–978

    Article  Google Scholar 

  • Tsai J.-F. (2007) An optimization approach for supply chain management models with quantity discount policy. European Jounal of Operational Research 177(2): 982–994

    Article  Google Scholar 

  • Vaario J., Ueda K. (1998) An emergent modelling method for dynamic scheduling. Journal of Intelligent Manufacturing 9: 129–140

    Article  Google Scholar 

  • Van Brussel H., Wyns J., Valckenaers P., Bongaerts L., Peeters L. (1998) Reference architecture for holonic manufacturing systems: PROSA. Computers in Industry 37(3): 255–274

    Article  Google Scholar 

  • Vilcot, G. (2007). Algorithmes approchés pour des problèmes d’ordonnancement multicritère de type job shop flexible et job shop multiressource, PhD thesis, University of François-Rabelais, Tours, France.

  • Voss S., Woodruff D. L. (2006) Introduction to computational optimization models for production planning in supply chain. Springer, Berlin

    Google Scholar 

  • Watkins, C. J. C. H. (1989). Learning from delayed rewards, PhD thesis, Cambridge University, Cambridge, England.

  • Weng Z. K., Wong R. T. (1993) General models for the supplier’s all-unit quantity discount policy. Naval Research Logistics 40(6): 971–991

    Article  Google Scholar 

  • Yang P. C. (2004) Pricing strategy for deteriorating items using quantity discount when demand is price sensitive. European Journal of Operational Research 157: 389–397

    Article  Google Scholar 

  • Zambrano, G., Aissani, N., Pach, C., Berger, T., & Trentesaux, D. (2011). An approach for temporal myopia reduction in heterarchical control architectures. In Proceedings of 20th IEEE inter symposium on industrial electronics, 27–30 June 2011.

  • Zbib, N., Pach, C., Sallez, Y., & Trentesaux, D. (2010). Heterarchical production control in manufacturing systems using the potential fields concept. Journal of Intelligent Manufacturing. doi:10.1007/s10845-010-0467-3.

  • Zhang H., Gen M. (2005) Multistage-based genetic algorithm for flexible job-shop scheduling problem. Journal of Complexity International 11: 223–232

    Google Scholar 

  • Zobolas G. I., Tarantilis C. D., Ioannou G. (2008) Exact, Heuristic and Meta-heuristic Algorithms for Solving Shop Scheduling Problems, Studies in Computational Intelligence (SCI). Springer, Berlin, pp 1–40

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LIO, Department of Computer Sciences, University of Oran, Oran, Algeria

    N. Aissani & B. Beldjilali

  2. University Lille Nord de France, 59000, Lille, France

    A. Bekrar & D. Trentesaux

  3. UVHC, TEMPO Lab., 59313, Valenciennes, France

    A. Bekrar & D. Trentesaux

Authors
  1. N. Aissani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Bekrar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Trentesaux
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. B. Beldjilali
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to N. Aissani.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aissani, N., Bekrar, A., Trentesaux, D. et al. Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning. J Intell Manuf 23, 2513–2529 (2012). https://doi.org/10.1007/s10845-011-0580-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-011-0580-y

Keywords

Search

Navigation