Skip to main content
SpringerLink
Log in

Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks

  • Published:
Journal of Intelligent Manufacturing Aims and scope Submit manuscript

Abstract

In this paper, we model preventive maintenance strategies for equipment composed of multi-non-identical components which have different time-to-failure probability distribution, by using a Markov decision process (MDP). The originality of this paper resides in the fact that a Monte Carlo reinforcement learning (MCRL) approach is used to find the optimal policy for each different strategy. The approach is applied to an already existing published application which deals with a fleet of military trucks. The fleet consists of a group of similar trucks that are composed of non-identical components. The problem is formulated as a MDP and solved by a MCRL technique. The advantage of this modeling technique when compared to the published one is that there is no need to estimate the main parameters of the model, for example the estimation of the transition probabilities. These parameters are treated as variables and they are found by the modeling technique, while searching for the optimal solution. Moreover, the technique is not bounded by any explicit mathematical formula, and it converges to the optimal solution whereas the previous model optimizes the replacement policy of each component separately, which leads to a local optimization. The results show that by using the reinforcement learning approach, we are able of getting a 36.44 % better solution that is less downtime.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Abdel Haleem, B., & Yacout, S. (1998). Simulation of components replacement policies for a fleet of military trucks. Quality Engineering, 11(2), 303–308.

    Article  Google Scholar 

  • Das, T. K., & Sarkar, S. (1999). Optimal preventive maintenance in a production inventory system. IIE Transactions, 31(6), 537–551.

    Google Scholar 

  • Gelly, S., Kocsis, L., Schoenauer, M., Sebag, M., Silver, D., Szepesvári, C., et al. (2012). The grand challenge of computer Go: Monte Carlo tree search and extensions. Communications of the ACM, 55(3), 106–113.

    Article  Google Scholar 

  • Gosavi, A. (2004). Reinforcement learning for long-run average cost. European Journal of Operational Research, 155(3), 654–674.

    Article  Google Scholar 

  • Jardine, A. K., & Tsang, A. H. (2013). Maintenance, replacement, and reliability: Theory and applications. Boca Raton: CRC Press.

    Book  Google Scholar 

  • Jia, Q.-S. (2010). A structural property of optimal policies for multi-component maintenance problems. IEEE Transactions on Automation Science and Engineering, 7(3), 677–680.

    Article  Google Scholar 

  • Powell, W. B. (2007). Approximate dynamic programming: Solving the curses of dimensionality (Vol. 703). New York: Wiley.

    Book  Google Scholar 

  • Steven, B. (2001). J. D. Campbell, A. K. Jardine, & W. M. Dekker (Eds.), Maintenance excellence, optimizing equipment life-cycle decisions, pp. 43–44.

  • Sutton, R. S., & Andrew, G. B. (1998). Reinforcement learning: An introduction (Vol. 1, No. 1). Cambridge: MIT press.

    Google Scholar 

  • Szepesvári, C. (2010). Algorithms for reinforcement learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 4(1), 1–103.

    Article  Google Scholar 

  • Tsitsiklis, J. N. (2003). On the convergence of optimistic policy iteration. The Journal of Machine Learning Research, 3, 59–72.

    Google Scholar 

  • Tuncel, E., Zeid, A., & Kamarthi, S. (2014). Solving large scale disassembly line balancing problem with uncertainty using reinforcement learning. Journal of Intelligent Manufacturing, 25(4), 647–659.

  • Wang, X., Wang, H., & Qi, C. (2014). Multi-agent reinforcement learning based maintenance policy for a resource constrained flow line system. Journal of Intelligent Manufacturing, 27(2), 325–333.

  • Wang, J. W., Wang, H., Ip, W. H., Furuta, K., & Zhang, W. J. (2013). Predatory search strategy based on swarm intelligence for continuous optimization problems. Mathematical Problems in Engineering. 11 pp. doi:10.1155/2013/749256

  • Zhang, W. J., & Van Luttervelt, C. A. (2011). Toward a resilient manufacturing system. CIRP Annals-Manufacturing Technology, 60(1), 469–472.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea

    Stephane R. A. Barde & Hayong Shin

  2. Ecole Polytechnique de Montreal, Montreal, QC, Canada

    Soumaya Yacout

Authors
  1. Stephane R. A. Barde
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Soumaya Yacout
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hayong Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Soumaya Yacout.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barde, S.R.A., Yacout, S. & Shin, H. Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks. J Intell Manuf 30, 147–161 (2019). https://doi.org/10.1007/s10845-016-1237-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10845-016-1237-7

Keywords

Search

Navigation