International Conference on Database and Expert Systems Applications

DEXA 2015: Database and Expert Systems Applications pp 153-161 | Cite as

An Efficient Gear-Shifting Power-Proportional Distributed File System

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9262)

Abstract

Recently, power-aware distributed file systems for efficient big data processing have increasingly moved toward power proportional designs. However, inefficient gear-shifting in such systems is an important issue that can seriously degrade their performance. To address this issue, we propose and evaluate an efficient gear-shifting power proportional distributed file system. The proposed system utilizes flexible data placement that reduces the amount of reflected data and has an architecture that improves the metadata management to achieve high-efficiency gear-shifting. Extensive empirical experiments using actual machines based on the HDFS demonstrated that the proposed system gains up to \(22\,\%\) better throughput-per-watt performance. Moreover, a suitable metadata management setting corresponding to the amount of data updated while in low gear is found from the experimental results.

References

  1. 1.
    André, B.L., Urs, H.: The case for energy-proportional computing. Computer 40, 33–37 (2007)Google Scholar
  2. 2.
    Charles, W., Mathew, O., Jin, Q., Andy, W.A.I., Peter, R., Geoff, K.: PARAID: a gear-shifting power-aware RAID. Trans. Storage 3(3), 13:1–13:33 (2007)Google Scholar
  3. 3.
    Thereska, E., Donnelly, A., Narayanan, D.: Sierra: practical power-proportionality for data center storage. In: Proceedings of 6th European Conference on Computer Systems, EuroSys 2011, pp. 169–182. ACM (2011)Google Scholar
  4. 4.
    Amur, H., Cipar, J., Gupta, V., Ganger, G.R., Kozuch, M.A., Schwan, K.: Robust and flexible power-proportional storage. In: Proceeding of the 1st ACM Symposium on Cloud Computing, SoCC 2010, pp. 217–228 (2010)Google Scholar
  5. 5.
    Le, H.H., Hikida, S., Yokota, H.: Efficient gear-shifting for a power-proportional distributed data-placement method. In: Proceedings 2013 IEEE International Conference on Big Data, pp. 76–84. IEEE (2013)Google Scholar
  6. 6.
    Le, H.H., Hikida, S., Yokota, H.: NDCouplingHDFS: a coupling architecture for a power-proportional hadoop distributed file system. IEICE Trans. Inf. Syst. E97–D(2), 213–222 (2014)CrossRefGoogle Scholar
  7. 7.
    Le, H.H., Hikida, S., Yokota, H.: Accordion: an efficient gear-shifting for a power-proportional distributed data-placement method. IEICE Trans. Inf. Syst. E98–D(5), 1013–1026 (2015)CrossRefMATHGoogle Scholar
  8. 8.
    Yokota, H., Kanemasa, Y., Miyazaki, J.: Fat-Btree: an update conscious parallel directory structure. In: Proceedings of the 15th International Conference on Data Engineering, ICDE 1999, pp. 448–457. IEEE Computer Society (1999)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Center for Technology InnovationR&D Group, Hitachi Ltd.KanagawaJapan
  2. 2.Department of Computer ScienceTokyo Institute of TechnologyMeguro, TokyoJapan

Personalised recommendations