Smart Manufacturing Systems: A Game Theory based Approach

  • Dorothea SchwungEmail author
  • Jan Niclas Reimann
  • Andreas Schwung
  • Steven X. Ding
Part of the Studies in Computational Intelligence book series (SCI, volume 864)


This paper presents a novel approach for self-optimization and learning as well as plug-and-play control of highly flexible, modular manufacturing units. The approach is inspired by recent encouraging results of game theory (GT) based learning in computer and control science. However, instead of representing the entire control behavior as a strategic game which might results in long training times and huge data set requirements, we restrict the learning process to the supervisor level by defining appropriate parameters from the basic control level (BCL) to be learned by learning agents. To this end, we define a set of interface parameters to the BCL programmed by IEC 61131 compatible code, which will be used for learning. Typical control parameters include switching thresholds, timing parameters and transition conditions. These parameters will then be considered as players in a multi-player game resulting in a distributed optimization approach. We apply the approach to a laboratory testbed consisting of different production modules which underlines the efficiency improvements for manufacturing units. In addition, plug-and-produce control is enabled by the approach as different configuration of production modules can efficiently be put in operation by re-learning the parameter sets.


  1. 1.
    J. Pfrommer, D. Stogl, K. Aleksandrov, S.E Navarro, B. Hein, J. Beyerer, Plug & produce by modelling skills and service-oriented orchestration of reconfigurable manufacturing systems. at-Automatisierungstechnik 63(10), pp. 790–800 (2015)Google Scholar
  2. 2.
    M. Schleipen, A. Lüder, O. Sauer, H. Flatt, J. Jasperneite, Requirements and concept for plug-and-work-adaptivity in the context of industry 4.0. at-Automatisierungstechnik 63(10), 801–820 (2015)Google Scholar
  3. 3.
    R.W. Brennan, P. Vrba, P. Tichy, A. Zoitl, C. Sünder, T. Strasser, V. Marik, Developments in dynamic and intelligent reconfiguration of industrial automation. Comput. Ind. 59(6), 533–547 (2008)CrossRefGoogle Scholar
  4. 4.
    P. Leitao, A. Walter Colombo, S. Karnouskos, Industrial automation based on cyber-physical systems technologies: prototype implementations and challenges, Comput. Ind. 81, 11–25 (2016)Google Scholar
  5. 5.
    J. Wang, Y. Ma, L. Zhang, R.X. Gao, D. Wu, Deep learning for smart manufacturing: methods and applications. J. Manufact. Syst. (in Press) (2018)Google Scholar
  6. 6.
    T. Wuest, D. Weimer, C. Irgens, K.-D. Thoben, Machine learning in manufacturing: advantages, challenges, and applications. Prod. Manufact. Res. 4(1), 23–45 (2016)CrossRefGoogle Scholar
  7. 7.
    D. Bauso, Game Theory with Engineering Applications (Society for Industrial and Applied Mathematics, Philadelphia, 2016)Google Scholar
  8. 8.
    S. Zazo, S. Valcarcel Macua, M. Sánchez-Fernández, J. Zazo, Dynamic potential games with constraints: fundamentals and applications in communications. IEEE Trans. Ind. Electron. Mag. 3(4), 49–55 (2015)Google Scholar
  9. 9.
    S. Buzzi, G. Colavolpe, D. Saturnino, A. Zappone, Potential games for energy-efficient power control and subcarrier allocation in uplink multicell OFDMA systems. IEEE J. Sel. Top. Sig. Process. 6(2), 89–103 (2012)CrossRefGoogle Scholar
  10. 10.
    K. Yamamoto, A comprehensive survey of potential game approaches to wireless networks. IEICE Trans. Commun. E98 B(9), 1804–1823 (2015)Google Scholar
  11. 11.
    J.R. Marden, S.D. Ruben, L.Y. Pao, A model-free approach to wind farm control using game theoretic methods. IEEE Trans. Control Syst. Technol. 21(4), 1207–1214 (2013)CrossRefGoogle Scholar
  12. 12.
    Q. Zhu, T. Basar, Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems. IEEE Control Syst. Mag. 35(1), 46–65 (2015)CrossRefGoogle Scholar
  13. 13.
    Y. Liang, F. Liu, S. Mei, Distributed real-time economic dispatch in smart grids: a state-based potential game approach. IEEE Trans. Smart Grids 21(4), 1207–1214 (2018)Google Scholar
  14. 14.
    Y. Zhao, S. Wang, T.C.E. Cheng, X. Yang, Z. Huang, Coordination of supply chains by option contracts: a cooperative game theory approach. Eur. J. Oper. Res. 207(2), 668–675 (2010)MathSciNetCrossRefGoogle Scholar
  15. 15.
    N. Li, D.W. Oyler, M. Zhang, Y. Yildiz, I. Kolmanovsky, A.R. Girard, Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans. Control Syst. Technol. 21(4), 1207–1214 (2019)Google Scholar
  16. 16.
    G. Zhou, P. Jiang, G.Q. Huang, A game-theory approach for job scheduling in networked manufacturing. Int. J. Adv. Manufact. Technol. 41(9–10), 972–985 (2009)CrossRefGoogle Scholar
  17. 17.
    A. Schwung, A. Elbel, D. Schwung, System reconfiguration of modular production units using a SOA-based control structure, in Proceedings of the 15th International Conference on Industrial Informatics (INDIN 2017), Emden, Germany (2017)Google Scholar
  18. 18.
    D. Schwung, J.N. Reimann, A. Schwung, S.X. Ding, Self learning in flexible manufacturing units: a reinforcement learning approach, in Proceedings of the 9th International Conference on Intelligent Systems, Funchal, Portugal (2018)Google Scholar
  19. 19.
    R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 1998)zbMATHGoogle Scholar
  20. 20.
    V. Mnih, A.P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu, Asynchronous methods for deep reinforcement learning, in Proceedings of The 33rd International Conference on Machine Learning, PMLR, 48 (2016), pp. 1928–1937Google Scholar
  21. 21.
    J.R. Marden, State based potential games. Automatica 48, 3075–3088 (2012)MathSciNetCrossRefGoogle Scholar
  22. 22.
    J.R. Marden, J.S. Shamma, Revisiting log-linear learning: asynchrony, completeness and payoff-based implementation. Games Econ. Behav. 75, 788–808 (2012)MathSciNetCrossRefGoogle Scholar
  23. 23.
    C. Lemke, M. Budka, B. Gabrys, Metalearning: a survey of trends and technologies. Artif. Intell. Rev. 44(1), 117–130 (2015)CrossRefGoogle Scholar
  24. 24.
    Programmable controllers part 3: programming languages. International Standard IEC 61131-3, 2nd edn. (2003)Google Scholar
  25. 25.
    Function blocks, International Standard IEC 61499, 1st edn. (2005)Google Scholar
  26. 26.
    T. Cucinotta, A. Mancina, G.F. Anastasi, G. Lipari, L. Mangeruca, R. Checcozzo, F. Rusina, A real-time service-oriented architecture for industrial automation. IEEE Trans. Ind. Inform. 5(3), 267–277 (2009)CrossRefGoogle Scholar
  27. 27.
    W. Dai, V. Vyatkin, J.H. Christensen, V.N. Dubinin, Bridging service-oriented architecture and IEC 61499 for flexibility and interoperability. IEEE Trans. Ind. Inf. 11(3), 771–781 (2015)CrossRefGoogle Scholar
  28. 28.
    A. Zoitl, T. Strasser, C. Sünder, T. Baier, Is IEC 61499 in harmony with IEC 61131–3? IEEE Ind. Electron. Mag. 3(4), 49–55 (2009)CrossRefGoogle Scholar
  29. 29.
    D. Schwung, T. Kempe, A. Schwung, Self-optimization of energy consumption in complex bulk good processes using reinforcement learning, in Proceedings of the 15th International Conference on Industrial Informatics (INDIN 2017), Emden, Germany (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Dorothea Schwung
    • 1
    Email author
  • Jan Niclas Reimann
    • 1
  • Andreas Schwung
    • 1
  • Steven X. Ding
    • 2
  1. 1.South Westphalia University of Applied SciencesSoestGermany
  2. 2.University of Duisburg-EssenDuisburgGermany

Personalised recommendations