Skip to main content
SpringerLink
Log in

A Deep Reinforcement Learning Approach for Large-Scale Service Composition

  • Conference paper
  • First Online:
PRIMA 2018: Principles and Practice of Multi-Agent Systems (PRIMA 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11224))

Abstract

As service-oriented environments become widespread, there exists a pressing need for service compositions to cope with the high scalability, complexity, heterogeneity and dynamicity features inherent in these environments. In this context, reinforcement learning has emerged as a powerful tool that empowers adaptive service composition in open and dynamic environments. However, most of the existing implementations of reinforcement learning algorithms for service compositions are inefficient and fail to handle large-scale service environments. Towards this end, this paper proposes a novel approach for adaptive service composition in dynamic and large-scale environments. The proposed approach employs deep reinforcement learning in order to address large-scale service environments with large number of service providers. Experimental results show the ability and efficiency of the proposed approach to provide successful service compositions in dynamic and large-scale service environments.

Supported by organization x.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Al-Masri, E., Mahmoud, Q.H.: Discovering the best web service. In: Proceedings of WWW, pp. 1257–1258 (2007)

    Google Scholar 

  2. Lee, C.-H., Hwang, S.-Y., Yen, I.-L.: A service pattern model for flexible service composition. In: Proceedings of IEEE ICWS, pp. 626–627, June 2012

    Google Scholar 

  3. Chiu, D., Agrawal, G.: Cost and accuracy aware scientific workflow composition for service-oriented environments. IEEE Trans. Serv. Comput. 4(2), 140–152 (2012)

    Google Scholar 

  4. Gelly, S., Silver, D.: Combining online and offline knowledge in UCT. In: Proceedings of ICML, ICML 2007, pp. 273–280. ACM, New York (2007)

    Google Scholar 

  5. Huang, J., Liu, Y., Yu, R., Duan, Q., Tanaka, Y.: Modeling and algorithms for QoS-aware service composition in virtualization-based cloud computing. IEICE Trans. 96–B(1), 10–19 (2013)

    Article  Google Scholar 

  6. Jureta, I.J., Faulkner, S., Achbany, Y., Saerens, M.: Dynamic web service composition within a service-oriented architecture. In: IEEE International Conference on Web Services, ICWS 2007, pp. 304–311, July 2007

    Google Scholar 

  7. Klein, A., Ishikawa, F., Honiden, S.: SanGA: a self-adaptive network-aware approach to service composition. IEEE Trans. Serv. Comput. 7(3), 452–464 (2014)

    Article  Google Scholar 

  8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Proceedings of NIPS, NIPS 2012, pp. 1097–1105. Curran Associates Inc. (2012)

    Google Scholar 

  9. Lawrence, S., Giles, C.L., Tsoi, A.C., Back, A.D.: Face recognition: a convolutional neural-network approach. IEEE Trans. Neural Netw. 8(1), 98–113 (1997)

    Article  Google Scholar 

  10. Li, D., Cheung, D., Shi, X., Ng, V.: Uncertainty reasoning based on cloud models in controllers. Comput. Math. Appl. 35(3), 99–123 (1998)

    Article  Google Scholar 

  11. Li, H., Dagli, C.H., Enke, D.: Short-term stock market timing prediction under reinforcement learning schemes. In: Proceedings of IEEE ADPRL, pp. 233–240 (2007)

    Google Scholar 

  12. Mnih, V.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  13. Moustafa, A., Zhang, M.: Towards proactive web service adaptation. In: Ralyté, J., Franch, X., Brinkkemper, S., Wrycza, S. (eds.) CAiSE 2012. LNCS, vol. 7328, pp. 473–485. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31095-9_31

    Chapter  Google Scholar 

  14. Moustafa, A., Zhang, M.: Multi-objective service composition using reinforcement learning. In: Basu, S., Pautasso, C., Zhang, L., Fu, X. (eds.) ICSOC 2013. LNCS, vol. 8274, pp. 298–312. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-45005-1_21

    Chapter  Google Scholar 

  15. Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. In: Proceedings of ICLR, Puerto Rico (2016)

    Google Scholar 

  16. Silver, D.: Mastering the game of go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)

    Article  Google Scholar 

  17. Tang, H., Liu, W., Zhou, L.: Web service composition method using hierarchical reinforcement learning. In: Yang, Y., Ma, M. (eds.) Green Communications and Networks. LNEE, vol. 113, pp. 1429–1438. Springer, Dordrecht (2012). https://doi.org/10.1007/978-94-007-2169-2_170

    Chapter  Google Scholar 

  18. Van Hasselt, H.: Double q-learning. In: Proceedings of NIPS, pp. 2613–2621. Curran Associates Inc. (2010)

    Google Scholar 

  19. Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of AAAI, AAAI 2016, pp. 2094–2100. AAAI Press (2016)

    Google Scholar 

  20. Wang, H., Chen, X., Wu, Q., Yu, Q., Zheng, Z., Bouguettaya, A.: Integrating on-policy reinforcement learning with multi-agent techniques for adaptive service composition. In: Franch, X., Ghose, A.K., Lewis, G.A., Bhiri, S. (eds.) ICSOC 2014. LNCS, vol. 8831, pp. 154–168. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45391-9_11

    Chapter  Google Scholar 

  21. Wang, H., Wang, X.: A novel approach to large-scale services composition. In: Ishikawa, Y., Li, J., Wang, W., Zhang, R., Zhang, W. (eds.) APWeb 2013. LNCS, vol. 7808, pp. 220–227. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37401-2_23

    Chapter  Google Scholar 

  22. Wang, H., Zhou, X., Zhou, X., Liu, W., Li, W., Bouguettaya, A.: Adaptive service composition based on reinforcement learning. In: Maglio, P.P., Weske, M., Yang, J., Fantinato, M. (eds.) ICSOC 2010. LNCS, vol. 6470, pp. 92–107. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17358-5_7

    Chapter  Google Scholar 

  23. Wang, S., Zheng, Z., Sun, Q., Zou, H., Yang, F.: Reliable web service selection via QoS uncertainty computing. Int. J. Web Grid Serv. 7(4), 410–426 (2011)

    Article  Google Scholar 

  24. Watkins, C.: Learning from delayed rewards. Ph.D. thesis, Cambridge University, England (1989)

    Google Scholar 

  25. Wenbo, X., Jian, C., Haiyan, Z., Lei, W.: A multi-agent learning model for service composition. In: Proceedings of APSCC, pp. 70–75, December 2012

    Google Scholar 

  26. Won Lee, J.: Stock price prediction using reinforcement learning. In: Proceedings of IEEE ISIE, vol. 1, pp. 690–695 (2001)

    Google Scholar 

  27. Ye, Z., Zhou, X., Bouguettaya, A.: Genetic algorithm based QoS-aware service compositions in cloud computing. In: Yu, J.X., Kim, M.H., Unland, R. (eds.) DASFAA 2011. LNCS, vol. 6588, pp. 321–334. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-20152-3_24

    Chapter  Google Scholar 

  28. Zuohua, D., Mingyue, J., Kandel, A.: Port-based reliability computing for service composition. IEEE Trans. Serv. Comput. 5(3), 422–436 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nagoya Instititute of Technology, Nagoya, Japan

    Ahmed Moustafa & Takayuki Ito

Authors
  1. Ahmed Moustafa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Takayuki Ito
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmed Moustafa .

Editor information

Editors and Affiliations

  1. School of Computing and Information Systems, University of Melbourne, Melbourne, Australia

    Tim Miller

  2. Department of Computer Science, University of Aberdeen, Aberdeen, UK

    Nir Oren

  3. Artificial Intelligence Research Center (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan

    Yuko Sakurai

  4. Artificial Intelligence Research Center (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Japan

    Itsuki Noda

  5. University of Otago, Dunedin, New Zealand

    Bastin Tony Roy Savarimuthu

  6. Computer Science Department, New Mexico State University, Las Cruces, USA

    Tran Cao Son

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Moustafa, A., Ito, T. (2018). A Deep Reinforcement Learning Approach for Large-Scale Service Composition. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds) PRIMA 2018: Principles and Practice of Multi-Agent Systems. PRIMA 2018. Lecture Notes in Computer Science(), vol 11224. Springer, Cham. https://doi.org/10.1007/978-3-030-03098-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03098-8_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03097-1

  • Online ISBN: 978-3-030-03098-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation