Production Engineering

, Volume 13, Issue 1, pp 33–41 | Cite as

Reinforcement learning for opportunistic maintenance optimization

  • Andreas KuhnleEmail author
  • Johannes Jakubik
  • Gisela Lanza
Production Management


Intelligent systems, that support the maintenance of production resources, offer real-time data-based approaches to optimize the maintenance effort and to reduce the usage of resources within production systems. However, unused potentials remain regarding maintenance schedules with minimal opportunity costs of the measures taken. This work provides a novel, machine-learning-based approach for the exploitation of these remaining optimization opportunities as an exemplary extension of the current state of the art. The determination of an optimal maintenance schedule for parallel working machines, is based on the data of a production system. The main result of this work is the performance of the implemented reinforcement learning algorithms, both in terms of downtime reduction, which increases the production output, and in terms of reducing maintenance costs compared to existing maintenance strategies. Hence, this work provides a holistic approach to the optimization of maintenance strategies and gives further evidence of a meaningful applicability of reinforcement learning algorithms in manufacturing processes.


Reinforcement learning Opportunistic maintenance Opportunity cost reduction Multi-agent-systems Proximal policy optimization Production planning and control 



We extend our sincere thanks to the German Federal Ministry of Education and Research (BMBF) for supporting this research project 02K16C082 Produktionsbezogene Dienstleistungssysteme auf Basis von Big-Data-Analysen (ProData).


  1. 1.
    Wuest T (2016) Machine learning in manufacturing: advantages, challenges, and applications. Prod Manuf Res 4:23-45Google Scholar
  2. 2.
    Colledani M, Magnanini MC, Tolio T (2018) Impact of opportunistic maintenance on manufacturing system performance. CIRP Ann 67(1):499–502CrossRefGoogle Scholar
  3. 3.
    Hashemian HM, Bean Wendell C (2011) State-of-the-art predictive maintenance techniques. IEEE Trans Instrum Meas 60(10):3480–3492CrossRefGoogle Scholar
  4. 4.
    Lindström J, Larsson H, Jonsson M, Leyon E (2017) Towards intelligent and sustainable production: combining and integratingonline predictive maintenance and continuous quality control. Procedia CIRP 63:443–448CrossRefGoogle Scholar
  5. 5.
    Yang L et al (2018) Opportunistic maintenance of production systems subject to random wait time and multiple control limits. J Manuf Syst 47:12–34CrossRefGoogle Scholar
  6. 6.
    Stricker N et al (2018) Reinforcement learning for adaptive order dispatching in the semiconductor industry. CIRP Ann 67(1):511–514CrossRefGoogle Scholar
  7. 7.
    Wang X et al (2014) Reinforcement learning based predictive maintenance for a machine with multiple deteriorating yield levels. J Comput Inf Syst 10(1):9–19Google Scholar
  8. 8.
    Wang J (2016) Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. J Intell Manuf 27(2):325–333CrossRefGoogle Scholar
  9. 9.
    Sutton RS, Barto AG (2017) Reinforcement learning: an introduction. MIT Press, CambridgeGoogle Scholar
  10. 10.
    Crites RH, Barto AG (1995) Improving elevator performance using reinforcement-learning. Adv Neural Inf Process Syst 8:1017–1023Google Scholar
  11. 11.
    Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M (2013) Playing atari with deep reinforcement learning. CoRR. arXiv:abs/1312.5602
  12. 12.
    Williams RJ (1992) Simple statistic gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256zbMATHGoogle Scholar
  13. 13.
    Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust region policy optimization. In: Proceedings of the 31st International Conference on Machine Learning, vol 37, pp 1889–1897Google Scholar
  14. 14.
    Schulman J (2017) Proximal policy optimization algorithms. Adv Neural Inf Process Syst 8:1017–1023Google Scholar
  15. 15.
    Schijve J (2009) Fatigue of structures and materials. Springer, Amsterdam, pp 15–21CrossRefzbMATHGoogle Scholar
  16. 16.
    Xie M, Lai CD (1996) Reliability analysis using an additive Weibull model with bathtub-shaped failure rate function. Reliab Eng Syst Saf 52(1):87–93CrossRefGoogle Scholar

Copyright information

© German Academic Society for Production Engineering (WGP) 2018

Authors and Affiliations

  1. 1.wbk, Institute of Production Science Karlsruhe Institute of Technology (KIT)KarlsruheGermany

Personalised recommendations