Abstract
Recently, power-aware distributed file systems for efficient big data processing have increasingly moved toward power proportional designs. However, inefficient gear-shifting in such systems is an important issue that can seriously degrade their performance. To address this issue, we propose and evaluate an efficient gear-shifting power proportional distributed file system. The proposed system utilizes flexible data placement that reduces the amount of reflected data and has an architecture that improves the metadata management to achieve high-efficiency gear-shifting. Extensive empirical experiments using actual machines based on the HDFS demonstrated that the proposed system gains up to \(22\,\%\) better throughput-per-watt performance. Moreover, a suitable metadata management setting corresponding to the amount of data updated while in low gear is found from the experimental results.
This work was done when the author was at Tokyo Institute of Technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
André, B.L., Urs, H.: The case for energy-proportional computing. Computer 40, 33–37 (2007)
Charles, W., Mathew, O., Jin, Q., Andy, W.A.I., Peter, R., Geoff, K.: PARAID: a gear-shifting power-aware RAID. Trans. Storage 3(3), 13:1–13:33 (2007)
Thereska, E., Donnelly, A., Narayanan, D.: Sierra: practical power-proportionality for data center storage. In: Proceedings of 6th European Conference on Computer Systems, EuroSys 2011, pp. 169–182. ACM (2011)
Amur, H., Cipar, J., Gupta, V., Ganger, G.R., Kozuch, M.A., Schwan, K.: Robust and flexible power-proportional storage. In: Proceeding of the 1st ACM Symposium on Cloud Computing, SoCC 2010, pp. 217–228 (2010)
Le, H.H., Hikida, S., Yokota, H.: Efficient gear-shifting for a power-proportional distributed data-placement method. In: Proceedings 2013 IEEE International Conference on Big Data, pp. 76–84. IEEE (2013)
Le, H.H., Hikida, S., Yokota, H.: NDCouplingHDFS: a coupling architecture for a power-proportional hadoop distributed file system. IEICE Trans. Inf. Syst. E97–D(2), 213–222 (2014)
Le, H.H., Hikida, S., Yokota, H.: Accordion: an efficient gear-shifting for a power-proportional distributed data-placement method. IEICE Trans. Inf. Syst. E98–D(5), 1013–1026 (2015)
Yokota, H., Kanemasa, Y., Miyazaki, J.: Fat-Btree: an update conscious parallel directory structure. In: Proceedings of the 15th International Conference on Data Engineering, ICDE 1999, pp. 448–457. IEEE Computer Society (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Le, H.H., Hikida, S., Yokota, H. (2015). An Efficient Gear-Shifting Power-Proportional Distributed File System. In: Chen, Q., Hameurlain, A., Toumani, F., Wagner, R., Decker, H. (eds) Database and Expert Systems Applications. Globe DEXA 2015 2015. Lecture Notes in Computer Science(), vol 9262. Springer, Cham. https://doi.org/10.1007/978-3-319-22852-5_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-22852-5_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-22851-8
Online ISBN: 978-3-319-22852-5
eBook Packages: Computer ScienceComputer Science (R0)