An Efficient Gear-Shifting Power-Proportional Distributed File System
Recently, power-aware distributed file systems for efficient big data processing have increasingly moved toward power proportional designs. However, inefficient gear-shifting in such systems is an important issue that can seriously degrade their performance. To address this issue, we propose and evaluate an efficient gear-shifting power proportional distributed file system. The proposed system utilizes flexible data placement that reduces the amount of reflected data and has an architecture that improves the metadata management to achieve high-efficiency gear-shifting. Extensive empirical experiments using actual machines based on the HDFS demonstrated that the proposed system gains up to \(22\,\%\) better throughput-per-watt performance. Moreover, a suitable metadata management setting corresponding to the amount of data updated while in low gear is found from the experimental results.
- 1.André, B.L., Urs, H.: The case for energy-proportional computing. Computer 40, 33–37 (2007)Google Scholar
- 2.Charles, W., Mathew, O., Jin, Q., Andy, W.A.I., Peter, R., Geoff, K.: PARAID: a gear-shifting power-aware RAID. Trans. Storage 3(3), 13:1–13:33 (2007)Google Scholar
- 3.Thereska, E., Donnelly, A., Narayanan, D.: Sierra: practical power-proportionality for data center storage. In: Proceedings of 6th European Conference on Computer Systems, EuroSys 2011, pp. 169–182. ACM (2011)Google Scholar
- 4.Amur, H., Cipar, J., Gupta, V., Ganger, G.R., Kozuch, M.A., Schwan, K.: Robust and flexible power-proportional storage. In: Proceeding of the 1st ACM Symposium on Cloud Computing, SoCC 2010, pp. 217–228 (2010)Google Scholar
- 5.Le, H.H., Hikida, S., Yokota, H.: Efficient gear-shifting for a power-proportional distributed data-placement method. In: Proceedings 2013 IEEE International Conference on Big Data, pp. 76–84. IEEE (2013)Google Scholar
- 8.Yokota, H., Kanemasa, Y., Miyazaki, J.: Fat-Btree: an update conscious parallel directory structure. In: Proceedings of the 15th International Conference on Data Engineering, ICDE 1999, pp. 448–457. IEEE Computer Society (1999)Google Scholar