Skip to main content
SpringerLink
Log in

Service Composition in Cloud Manufacturing: A DQN-Based Approach

  • Chapter
  • First Online:
Scheduling in Industry 4.0 and Cloud Manufacturing

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 289))

Abstract

Cloud manufacturing is a new service-oriented manufacturing model that integrates distributed manufacturing resources to provide on-demand manufacturing services over the Internet. Service composition that builds larger-granularity, value-added services by combining a number of smaller-granularity services to satisfy consumers’ complex requirements is an important issue in cloud manufacturing. Meta-heuristic algorithms such as genetic algorithm, particle swarm optimization, and ant colony algorithm are frequently employed for addressing service composition issues in cloud manufacturing. However, these algorithms require complex design flows and lack adaptability to dynamic environment. Deep reinforcement learning provides an alternative approach for solving cloud manufacturing service composition issues. This chapter proposes a deep Q-network (DQN) based approach for service composition in cloud manufacturing, which is able to find optimal service composition solutions through repeated training and learning. Results of experiments that take into account changes of service scales and service unavailability reveal the scalability and robustness of the DQN algorithm-based service composition approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Chen, J., Huang, G. Q., Wang, J., & Yang, C. (2019). A cooperative approach to service booking and scheduling in cloud manufacturing. European Journal of Operational Research, 273(3), 861–873.

    Article  Google Scholar 

  • Christiano, P. F., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017). Deep reinforcement learning from human preferences. arXiv preprint arXiv:1706.03741v3.

    Google Scholar 

  • de Bruin, T., Kober, J., Tuyls, K., & Babuška, R. (2015). The importance of experience replay database composition in deep reinforcement learning. In Deep reinforcement learning workshop, NIPS.

    Google Scholar 

  • Ding, T., Yan, G., Lei, Y., & Xiangyu, X. (2020). A niching behaviour-based algorithm for multi-level manufacturing service composition optimal-selection. Journal of Ambient Intelligence and Humanized Computing, 11, 1177–1189.

    Article  Google Scholar 

  • Dolgui, A., Ivanov, D., Sethi, S. P., & Sokolov, B. (2019). Scheduling in production, supply chain and Industry 4.0 systems by optimal control: Fundamentals, state-of-the-art and applications. International Journal of Production Research, 57(2), 411–432.

    Article  Google Scholar 

  • Duflou, J. R., Sutherland, J. W., Dornfeld, D., Herrmann, C., Jeswiet, J., Kara, S., Hauschild, M., & Kellens, K. (2012). Towards energy and resource efficient manufacturing: A processes and systems approach. CIRP Annals, 61(2), 587–609.

    Article  Google Scholar 

  • Ghomi, E. J., Rahmani, A. M., & Qader, N. N. (2019). Service load balancing, scheduling, and logistics optimization in cloud manufacturing by using genetic algorithm. Concurrency and Computation: Practice and Experience, 31(20), e5329.

    Google Scholar 

  • Ivanov, D., Dolgui, A., Sokolov, B., Werner, F., & Ivanova, M. (2016). A dynamic model and an algorithm for short-term supply chain scheduling in the smart factory industry 4.0. International Journal of Production Research, 54(2), 386–402.

    Article  Google Scholar 

  • Jackson, P. C. (2019). Introduction to artificial intelligence. New York: Courier Dover Publications.

    Google Scholar 

  • Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.

    Google Scholar 

  • Li, C., Guan, J., Liu, T., Ma, N., & Zhang, J. (2018). An autonomy-oriented method for service composition and optimal selection in cloud manufacturing. The International Journal of Advanced Manufacturing Technology, 96(5–8), 2583–2604.

    Article  Google Scholar 

  • Li, Y., Yao, X., & Liu, M. (2019). Cloud manufacturing service composition optimization with improved genetic algorithm. Mathematical Problems in Engineering, 2019, 7194258.

    Google Scholar 

  • Lin, Z., Luo, Y., Tao, F., Li, B. H., Ren, L., Zhang, X., Guo, H., Cheng, Y., Hu, A., & Liu, Y. (2014). Cloud manufacturing: A new manufacturing paradigm. Enterprise Information Systems, 8(2), 167–187.

    Article  Google Scholar 

  • Liu, W., Ma, G., & Liu, B. (2013). Study on hierarchical service composition in cloud manufacturing. China Mechanical Engineering, 24(10), 1349–1356.

    Google Scholar 

  • Liu, Y., Xu, X., Srinivasan, A., & Zhang, L. (2017). Enterprises in cloud manufacturing: A preliminary exploration. In ASME 2017 12th International Manufacturing Science and Engineering Conference collocated with the JSME/ASME 2017 6th International Conference on Materials and Processing, June 4–8, 2017, Los Angeles, CA, USA.

    Google Scholar 

  • Liu, Y., Wang, L., & Wang, X. V. (2018). Cloud manufacturing: Latest advancements and future trends. Procedia Manufacturing, 25, 62–73.

    Article  Google Scholar 

  • Liu, Y., Wang, L., Wang, X. V., Xu, X., & Jiang, P. (2019a). Cloud manufacturing: Key issues and future perspectives. International Journal of Computer Integrated Manufacturing, 32(9), 858–874.

    Article  Google Scholar 

  • Liu, Y., Wang, L., Wang, X. V., Xu, X., & Lin, Z. (2019b). Scheduling in cloud manufacturing: State-of-the-art and research challenges. International Journal of Production Research, 57(15–16), 4854–4879.

    Article  Google Scholar 

  • Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M., et al. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602.

    Google Scholar 

  • Mnih, V., Kavukcuoglu, K., Silver, D., et al. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529–533.

    Article  Google Scholar 

  • Panetto, H., Iung, B., Ivanov, D., Weichhart, G., & Wang, X. (2019). Challenges for the cyber-physical manufacturing enterprises of the future. Annual Reviews in Control, 47, 200–213.

    Article  Google Scholar 

  • Pisching, M. A., Junqueira, F., Santos Filho, D. J., & Miyagi, P. E. (2015). Service composition in the cloud-based manufacturing focused on the industry 4.0. In Doctoral Conference on Computing, Electrical and Industrial Systems (pp. 65–72). Cham: Springer.

    Google Scholar 

  • Silver, D., Huang, A., Maddison, C. J., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489.

    Article  Google Scholar 

  • Tao, F., Lin, Z., Guo, H., Luo, Y., & Ren, L. (2011). Typical characteristics of cloud manufacturing and several key issues of cloud service composition. Computer Integrated Manufacturing Systems, 17(3), 477–486.

    Google Scholar 

  • Tao, F., Lin, Z., Liu, Y., Cheng, Y., Wang, L., & Xun, X. (2015). Manufacturing service management in cloud manufacturing: Overview and future research directions. Journal of Manufacturing Science and Engineering, 137(4), 040912.

    Article  Google Scholar 

  • Vidayev, I. G., Martyushev, N. V., Ivashutenko, A. S., & Bogdan, A. M. (2014). The resource efficiency assessment technique for the foundry production. Advanced Materials Research, 880, 141–145.

    Article  Google Scholar 

  • Wang, T., Guo, S., & Lee, C.-G. (2014). Manufacturing task semantic modeling and description in cloud manufacturing system. The International Journal of Advanced Manufacturing Technology, 71(9–12), 2017–2031.

    Article  Google Scholar 

  • Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292.

    Google Scholar 

  • Xiang, F., Jiang, G., Xu, L., & Wang, N. (2016). The case-library method for service composition and optimal selection of big manufacturing data in cloud manufacturing system. The International Journal of Advanced Manufacturing Technology, 84(1–4), 59–70.

    Article  Google Scholar 

  • Xu, L., Jiawei, D., & Ming, H. (2017). Research on hybrid cloud particle swarm optimization for multi-objective flexible job shop scheduling problem. In 2017 IEEE 6th International Conference on Computer Science and Network Technology (ICCSNT), October 21–22, 2017, Dalian, China.

    Google Scholar 

  • Yang, Y., Yang, B., Wang, S., Liu, F., Wang, Y., & Shu, X. (2019). A dynamic ant-colony genetic algorithm for cloud service composition optimization. The International Journal of Advanced Manufacturing Technology, 102(1–4), 355–368.

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Natural Science Foundation of China (NSFC) under Grant Nos. 61973243 and 61873014.

Author information

Authors and Affiliations

  1. School of Electronics and Control, Chang’an University, Xi’an, China

    Haifeng Zhang & Huagang Liang

  2. School of Mechano-Electronic Engineering, Xidian University, Xi’an, China

    Yongkui Liu

  3. Department of Production Engineering, KTH Royal Institute of Technology, Stockholm, Sweden

    Lihui Wang

  4. School of Automation Science and Electrical Engineering, Beihang University, Beijing, China

    Lin Zhang

Authors
  1. Haifeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongkui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huagang Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lihui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lihui Wang .

Editor information

Editors and Affiliations

  1. Saint Petersburg Institute for Informatics and Automation of the RAS (SPIIRAS), St. Petersburg, Russia

    Boris Sokolov

  2. Berlin School of Economics and Law, Berlin, Germany

    Dmitry Ivanov

  3. IMT Atlantique, LS2N—CNRS, La Chantrerie, Nantes, France

    Alexandre Dolgui

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, H., Liu, Y., Liang, H., Wang, L., Zhang, L. (2020). Service Composition in Cloud Manufacturing: A DQN-Based Approach. In: Sokolov, B., Ivanov, D., Dolgui, A. (eds) Scheduling in Industry 4.0 and Cloud Manufacturing. International Series in Operations Research & Management Science, vol 289. Springer, Cham. https://doi.org/10.1007/978-3-030-43177-8_12

Download citation

Publish with us

Policies and ethics

Search

Navigation