A Reinforcement Learning Based Placement Strategy in Datacenter Networks

  • Weihong Yang
  • Yang QinEmail author
  • ZhaoZheng Yang
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 300)


As the core infrastructure of cloud computing, the datacenter networks place heavy demands on efficient storage and management of massive data. Data placement strategy, which decides how to assign data to nodes for storage, has a significant impact on the performance of the datacenter. However, most of the existing solutions cannot be better adaptive to the dynamics of the network. Moreover, they focus on where to store the data (i.e., the selection of storage node) but have not considered how to store them (i.e., the selection of routing path). Since reinforcement learning (RL) has been developed as a promising solution to address dynamic network issues, in this paper, we integrate RL into the datacenter networks to deal with the data placement issue. Considering the dynamics of resources, we propose a Q-learning based data placement strategy for datacenter networks. By leveraging Q-learning, each node can adaptively select next-hop based on the network information collected from downstream, and forward the data toward the storage node that has adequate capacity along the path with high available bandwidth. We evaluate our proposal on the NS-3 simulator in terms of average delay, throughput, and load balance. Simulation results show that the Q-learning placement strategy can effectively reduce network delay and increase average throughout while achieving load-balanced among servers.


Datacenter networks Placement strategy Q-learning 


  1. 1.
    Xia, W., Zhao, P., Wen, Y., Xie, H.: A survey on data center networking (DCN): infrastructure and operations. IEEE Commun. Surv. Tutor. 19, 640–656 (2017). Scholar
  2. 2.
    Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles, Bolton Landing, NY, pp. 20–43 (2003)Google Scholar
  3. 3.
    Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44, 35–40 (2010). Scholar
  4. 4.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10 (2010).
  5. 5.
    Renuga, K., Tan, S.S., Zhu, Y.Q., Low, T.C., Wang, Y.H.: Balanced and efficient data placement and replication strategy for distributed backup storage systems. In: 2009 International Conference on Computational Science and Engineering, pp. 87–94 (2009).
  6. 6.
    Zaman, S., Grosu, D.: A distributed algorithm for the replica placement problem. IEEE Trans. Parallel Distrib. Syst. 22, 1455–1468 (2011). Scholar
  7. 7.
    Rajalakshmi, A., Vijayakumar, D., Srinivasagan, K.G.: An improved dynamic data replica selection and placement in cloud. In: 2014 International Conference on Recent Trends in Information Technology, pp. 1–6 (2014).
  8. 8.
    Vilaça, R., Oliveira, R., Pereira, J.: A correlation-aware data placement strategy for key-value stores. In: Felber, P., Rouvoy, R. (eds.) DAIS 2011. LNCS, vol. 6723, pp. 214–227. Springer, Heidelberg (2011). Scholar
  9. 9.
    Meroufel, B., Belalem, G.: Dynamic replication based on availability and popularity in the presence of failures. J. Inf. Process. Syst. 8, 263–278 (2012)CrossRefGoogle Scholar
  10. 10.
    Paiva, J., Ruivo, P., Romano, P., Rodrigues, L.: AutoPlacer: scalable self-tuning data placement in distributed key-value stores. ACM Trans. Auton. Adapt. Syst. (TAAS) 9, 19 (2015)Google Scholar
  11. 11.
    Wu, J.-J., Lin, Y.-F., Liu, P.: Optimal replica placement in hierarchical Data Grids with locality assurance. J. Parallel Distrib. Comput. 68, 1517–1538 (2008)CrossRefGoogle Scholar
  12. 12.
    Gao, C., Wang, H., Zhai, L., Gao, Y., Yi, S.: An energy-aware ant colony algorithm for network-aware virtual machine placement in cloud computing. In: 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), pp. 669–676. IEEE (2016)Google Scholar
  13. 13.
    Lian, Q., Chen, W., Zhang, Z.: On the impact of replica placement to the reliability of distributed brick storage systems. In: 25th IEEE International Conference on Distributed Computing Systems (ICDCS 2005), pp. 187–196 (2005).
  14. 14.
    Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: controlled, scalable, decentralized placement of replicated data. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC 2006, p. 31 (2006).
  15. 15.
    Al-Fares, M., Loukissas, A., Vahdat, A.: A scalable, commodity data center network architecture. In: Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication, pp. 63–74. ACM, New York (2008).
  16. 16.
    Doltsinis, S., Ferreira, P., Lohse, N.: An MDP model-based reinforcement learning approach for production station ramp-up optimization: q-learning analysis. IEEE Trans. Syst. Man Cybern.: Syst. 44, 1125–1138 (2014). Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceHarbin Institute of Technology (Shenzhen)ShenzhenChina

Personalised recommendations