Abstract
As the core infrastructure of cloud computing, the datacenter networks place heavy demands on efficient storage and management of massive data. Data placement strategy, which decides how to assign data to nodes for storage, has a significant impact on the performance of the datacenter. However, most of the existing solutions cannot be better adaptive to the dynamics of the network. Moreover, they focus on where to store the data (i.e., the selection of storage node) but have not considered how to store them (i.e., the selection of routing path). Since reinforcement learning (RL) has been developed as a promising solution to address dynamic network issues, in this paper, we integrate RL into the datacenter networks to deal with the data placement issue. Considering the dynamics of resources, we propose a Q-learning based data placement strategy for datacenter networks. By leveraging Q-learning, each node can adaptively select next-hop based on the network information collected from downstream, and forward the data toward the storage node that has adequate capacity along the path with high available bandwidth. We evaluate our proposal on the NS-3 simulator in terms of average delay, throughput, and load balance. Simulation results show that the Q-learning placement strategy can effectively reduce network delay and increase average throughout while achieving load-balanced among servers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xia, W., Zhao, P., Wen, Y., Xie, H.: A survey on data center networking (DCN): infrastructure and operations. IEEE Commun. Surv. Tutor. 19, 640–656 (2017). https://doi.org/10.1109/COMST.2016.2626784
Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google file system. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles, Bolton Landing, NY, pp. 20–43 (2003)
Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44, 35–40 (2010). https://doi.org/10.1145/1773912.1773922
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–10 (2010). https://doi.org/10.1109/MSST.2010.5496972
Renuga, K., Tan, S.S., Zhu, Y.Q., Low, T.C., Wang, Y.H.: Balanced and efficient data placement and replication strategy for distributed backup storage systems. In: 2009 International Conference on Computational Science and Engineering, pp. 87–94 (2009). https://doi.org/10.1109/CSE.2009.27
Zaman, S., Grosu, D.: A distributed algorithm for the replica placement problem. IEEE Trans. Parallel Distrib. Syst. 22, 1455–1468 (2011). https://doi.org/10.1109/TPDS.2011.27
Rajalakshmi, A., Vijayakumar, D., Srinivasagan, K.G.: An improved dynamic data replica selection and placement in cloud. In: 2014 International Conference on Recent Trends in Information Technology, pp. 1–6 (2014). https://doi.org/10.1109/ICRTIT.2014.6996180
Vilaça, R., Oliveira, R., Pereira, J.: A correlation-aware data placement strategy for key-value stores. In: Felber, P., Rouvoy, R. (eds.) DAIS 2011. LNCS, vol. 6723, pp. 214–227. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21387-8_17
Meroufel, B., Belalem, G.: Dynamic replication based on availability and popularity in the presence of failures. J. Inf. Process. Syst. 8, 263–278 (2012)
Paiva, J., Ruivo, P., Romano, P., Rodrigues, L.: AutoPlacer: scalable self-tuning data placement in distributed key-value stores. ACM Trans. Auton. Adapt. Syst. (TAAS) 9, 19 (2015)
Wu, J.-J., Lin, Y.-F., Liu, P.: Optimal replica placement in hierarchical Data Grids with locality assurance. J. Parallel Distrib. Comput. 68, 1517–1538 (2008)
Gao, C., Wang, H., Zhai, L., Gao, Y., Yi, S.: An energy-aware ant colony algorithm for network-aware virtual machine placement in cloud computing. In: 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS), pp. 669–676. IEEE (2016)
Lian, Q., Chen, W., Zhang, Z.: On the impact of replica placement to the reliability of distributed brick storage systems. In: 25th IEEE International Conference on Distributed Computing Systems (ICDCS 2005), pp. 187–196 (2005). https://doi.org/10.1109/ICDCS.2005.56
Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: controlled, scalable, decentralized placement of replicated data. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC 2006, p. 31 (2006). https://doi.org/10.1109/SC.2006.19
Al-Fares, M., Loukissas, A., Vahdat, A.: A scalable, commodity data center network architecture. In: Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication, pp. 63–74. ACM, New York (2008). https://doi.org/10.1145/1402958.1402967
Doltsinis, S., Ferreira, P., Lohse, N.: An MDP model-based reinforcement learning approach for production station ramp-up optimization: q-learning analysis. IEEE Trans. Syst. Man Cybern.: Syst. 44, 1125–1138 (2014). https://doi.org/10.1109/TSMC.2013.2294155
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Yang, W., Qin, Y., Yang, Z. (2020). A Reinforcement Learning Based Placement Strategy in Datacenter Networks. In: Chu, X., Jiang, H., Li, B., Wang, D., Wang, W. (eds) Quality, Reliability, Security and Robustness in Heterogeneous Systems. QShine 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 300. Springer, Cham. https://doi.org/10.1007/978-3-030-38819-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-38819-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38818-8
Online ISBN: 978-3-030-38819-5
eBook Packages: Computer ScienceComputer Science (R0)