Advertisement

Journal of Intelligent Manufacturing

, Volume 25, Issue 4, pp 647–659 | Cite as

Solving large scale disassembly line balancing problem with uncertainty using reinforcement learning

  • Emre Tuncel
  • Abe ZeidEmail author
  • Sagar Kamarthi
Article

Abstract

Due to increasing environmental concerns, manufacturers are forced to take back their products at the end of products’ useful functional life. Manufacturers explore various options including disassembly operations to recover components and subassemblies for reuse, remanufacture, and recycle to extend the life of materials in use and cut down the disposal volume. However, disassembly operations are problematic due to high degree of uncertainty associated with the quality and configuration of product returns. In this research we address the disassembly line balancing problem (DLBP) using a Monte-Carlo based reinforcement learning technique. This reinforcement learning approach is tailored fit to the underlying dynamics of a DLBP. The research results indicate that the reinforcement learning based method is able to perform effectively, even on a complex large scale problem, within a reasonable amount of computational time. The proposed method performed on par or better than the benchmark methods for solving DLBP reported in the literature. Unlike other methods which are usually limited deterministic environments, the reinforcement learning based method is able to operate in deterministic as well as stochastic environments.

Keywords

Disassembly Reinforcement learning Heuristics  Disassembly line balancing Cell phone PC 

References

  1. Agarwal, S., & Tiwari, M. K. (2006). A collaborative ant colony algorithm to stochastic mixed-model U-shaped disassembly line balancing and sequencing problem. International Journal of Production, 46, 1405–1429.CrossRefGoogle Scholar
  2. Aissani, N., Bekrar, A., & et al. (2011). Dynamic scheduling for multi-site companies: a decisional approach based on reinforcement multi-agent learning. Journal of Intelligent Manufacturing: in press.Google Scholar
  3. Altekin, F. T., Kandiller, L., et al. (2008). Profit-oriented disassembly line balancing. International Journal of Production Research, 46, 2675–2693.Google Scholar
  4. Banda, K., & Zeid, I. (2006). To disassemble or not: A computational methodology for decision making. Journal of Intelligent Manufacturing, 17(5), 621–634.CrossRefGoogle Scholar
  5. Giudice, F., & Fargione, G. (2007). Disassembly planning of mechanical systems for service and recovery: A genetic algorithms based approach. Journal of Intelligent Manufacturing, 18(3), 313–329.CrossRefGoogle Scholar
  6. Gungor, A., & Gupta, S. (2002). Disassembly line in product recovery. International Journal of Production Research, 40(11), 2569–2589.CrossRefGoogle Scholar
  7. Gungor, A., & Gupta, S. M. (1999a). Disassembly line balancing. In Proceedings of the 1999 annual meeting of the northeast decision sciences, RI, Newport.Google Scholar
  8. Gungor, A., & Gupta, S. M. (1999b). Issues in environmentally concious manufacturing and product recovery: A survey. Computers and Industrial Engineering, 36, 811–853.CrossRefGoogle Scholar
  9. Gungor, A., Gupta, S. M., & et al. (2001). Complications in disassembly line balancing. In Proceedings of SPIE—the international society for optical engineering, SPIE.Google Scholar
  10. Gupta, S. M., Erbis, E., & et al. (2004). Disassembly sequencing problem: A case study of a cell phone. In Proceedings of SPIE—The international society for optical engineering, SPIE.Google Scholar
  11. Gupta, S. M., & Gungor, A. (2001). Product recovery using a disassembly line: Challenges and solution. In IEEE international symposium on electronics and the environment, Institute of Electrical and Electronics Engineers Inc.Google Scholar
  12. Gupta, S. M., & Lambert, A. J. D. (2008). Environment conscious manufacturing. Boca Raton: CRC Press.Google Scholar
  13. Homem de Mello, L. S., & Sanderson, A. C. (1990). AND/OR graph representation of assembly plans. IEEE Transactions on Robotics and Automation, 6(2), 188–199.CrossRefGoogle Scholar
  14. Kizilkaya, E. A., & Gupta, S. M. (2005). Impact of different disassembly line balancing algorithms on the performance of dynamic kanban system for disassembly line. In Proceedings of the SPIE—the international society for optical engineering, SPIE, USA.Google Scholar
  15. Kongar, E., & Gupta, S. M. (2006). Disassembly sequencing using genetic algorithm. Internationl Journal of Advanced Manufacturing Technology, 30(5–6), 497–506.CrossRefGoogle Scholar
  16. Lambert, A. J. D. (2001). Optimum disassembly sequence generation. Environmentally conscious manufacturing. In Proceedings of SPIE—the international society for optical engineering, Vol. 4193, pp. 56–67.Google Scholar
  17. Lambert, A. J. D. (2007). Optimizing disassembly processes subjected to sequence-dependent cost. Computers and Operations Research, 34(2), 536–551.CrossRefGoogle Scholar
  18. Lambert, A. J. D., Gupta, S. M. (2005a). Determining optimum and suboptimum disassembly sequences with an application to a cell phone. In Proceedings of the IEEE international symposium on assembly and task planning, Institute of Electrical and Electronics Engineers Computer Society.Google Scholar
  19. Lambert, A. J. D., & Gupta, S. M. (2005b). Disassembly modeling for assembly, maintenance, and reuse. New York: CRC.Google Scholar
  20. Martinez, M., Pham, F., et al. (2009). Optimal assembly plan generation: A simplifying approach. Journal of Intelligent Manufacturing, 20(1), 15–27.Google Scholar
  21. McGovern, S., Gupta, S. M. (2004a) Combinatorial optimization methods for disassembly line balancing. In Proceedings of the 2004 SPIE international conference on enviromentally conscious manufacturing, Philadelphia.Google Scholar
  22. McGovern, S., & Gupta, S. M. (2007a). A balancing method and genetic algorithm for disassembly line balancing. European Journal of Operational Research, 179, 692–708.CrossRefGoogle Scholar
  23. McGovern, S. M., & Gupta, S. M. (2004b). 2-Opt Heuristic for the Disassembly Line Balancing Problem. In Proceedings of SPIE—the international society for optical engineering, SPIE. Google Scholar
  24. McGovern, S. M., & Gupta, S. M. (2007b). Combinatorial optimization analysis of unary NP-complete disassembly line balancing problem. International Journal of Production Research, 45(18–19), 4485–4511.Google Scholar
  25. McGovern, S. M., & Gupta, S. M. (2011). The disassembly line—balancing and modeling. New York City: McGraw-Hill.Google Scholar
  26. Pan, L., & Zeid, I. (2001). A knowledge base for indexing and retrieving disassembly plans. Journal of Intelligent Manufacturing, 12(1), 77–94.CrossRefGoogle Scholar
  27. Reveliotis, S. A. (2007). Uncertainty management in optimal disassembly planning through learning-based strategies. IIE Transactions, 39(6), 645–658.CrossRefGoogle Scholar
  28. Russell, S., & Norvig, P. (1995). Artificial intelligence: A modern approach. Eaglewood Cliffs, New Jersey: Prentice Hall.Google Scholar
  29. Seo, K.-K., Park, J.-H., et al. (2001). Optimal disassembly sequence using genetic algorithms considering economic and environmental aspects. International Journal of Advanced Manufacturing Technology, 18, 371–380.Google Scholar
  30. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, Massachusetts: The MIT Press.Google Scholar
  31. Tadao, M. (1989). Petri nets: properties, analysis and applications. In Proceedings of the IEEE.Google Scholar
  32. Tang, Y., & MengChu, Z. (2006). A systematic approach to design and operation of disassembly lines. IEEE Transactions on Automation Science and Engineering, 3(3), 324–329.CrossRefGoogle Scholar
  33. Tang, Y., Zhou, M., et al. (2002). Disassembly modeling, plannig, and application. Journal of Manufacturing Systems, 2(3), 200–217.Google Scholar
  34. Tewari, A. (2007). Reinforcement learning in large or unknown MDPs. Ph.D: Dissertation, University of California, Berkeley.Google Scholar
  35. Turowski, M., Morgan, M., & et al. (2005). Disassembly line design with uncertainty. In Conference proceedings—IEEE international conference on systems, man and cybernetics, Institute of Electrical and Electronics Engineers Inc.Google Scholar
  36. Veerakamolmal, P., & Gupta, S. M. (2002). A case-based reasoning approach for automating disassembly process planning. Journal of Intelligent Manufacturing, 13(1), 47–60.CrossRefGoogle Scholar
  37. Watkins, C., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3), 279–292.Google Scholar
  38. Watkins, C. J. C. H. (1989). Learning from Delayed Rewards. Ph.D. thesis. U.K., Cambridge University.Google Scholar
  39. Zeid, I., Gupta, S., et al. (1997). A case-based reasoning approach to planning for disassembly. Journal of Intelligent Manufacturing, 8(2), 97–106.Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.Department of Mechanical and Industrial EngineeringNortheastern UniversityBostonUSA

Personalised recommendations