The Journal of Supercomputing

, Volume 73, Issue 12, pp 5465–5495 | Cite as

Model-based energy-aware data movement optimization in the storage I/O stack

  • Pablo Llopis
  • Florin Isaila
  • Javier Garcia Blas
  • Jesus Carretero


The increasing data demands of applications from various domains and the decreasing relative power cost of CPU computation have gradually exposed data movement cost as the prominent factor of energy consumption in computing systems. The traditional organization of the computer system software into a layered stack, while providing a straightforward modularity, poses a significant challenge for the global optimization of data movement in particular and, thus, the energy efficiency in general. Optimizing the energy efficiency of data movement in large-scale systems is a difficult tasks because it depends on a complex interplay of various factors at different system layers. In this work, we address the challenge of optimizing the data movement of the storage I/O stack in a holistic manner. Our approach consists of a model-based system driver that obtains the current I/O power regime and adapts the CPU frequency level according to this information. On the one hand, for simplifying the understanding of the relation between data movement and energy efficiency, this paper proposes novel energy prediction models for data movement based on series of runtime metrics from several I/O stack layers. We provide an in-depth study of the energy consumption in the data path, including the identification and analysis of power and performance regimes that synthesize the energy consumption patterns in a cross-layer approach. On the other hand, we propose and prototype a kernel driver that exploits data movement awareness for improving the current CPU-centric energy management.


Storage Storage I/O Energy efficiency Power usage Statistical analysis 



This work has been partially supported through grants TIN2016-79637-P “Towards unification of HPC and Big Data Paradigms” from the Spanish Ministry of Economy and Competitiveness.


  1. 1.
    Kogge P, Bergman K, Borkar S, Campbell D, Carson W, Dally W, Denneau M, Franzon P, Harrod W, Hill K et al Exascale computing study: technology challenges in achieving exascale systemsGoogle Scholar
  2. 2.
    Borkar S, Chien AA (2011) The future of microprocessors. Commun ACM 54(5):67–77CrossRefGoogle Scholar
  3. 3.
    Reed DA, Dongarra J (2015) Exascale computing and big data. Commun ACM 58(7):56–68CrossRefGoogle Scholar
  4. 4.
    Llopis P, Dolz MF, Blas JG, Isaila F, Heidari MR, Kuhn M (2016) Analyzing the energy consumption of the storage data path. J Supercomput 72:4089–4106CrossRefGoogle Scholar
  5. 5.
    Axboe J Flexible I/O tester.
  6. 6.
    Ge R, Feng X, Song S, Chang H-C, Li D, Cameron KW (2010) Powerpack: energy profiling and analysis of high-performance systems and applications. IEEE Trans Parallel Distrib Syst 21(5):658–671CrossRefGoogle Scholar
  7. 7.
    Dolz MF, Heidari MR, Kuhn M, Fabregat G (2015) ArduPower: a low-cost wattmeter to improve energy efficiency of HPC applications. In: 5th International Green & Sustainable Computing Conference, Las Vegas, NV, USAGoogle Scholar
  8. 8.
    Barrachina S, Barreda M, Catalán S, Dolz M, Fabregat G, Mayo R, Quintana-Ortí E (2013) An integrated framework for power-performance analysis of parallel scientific workloads. In: ENERGY 2013, The 3rd International Conference on Smart Grids, Green Communications and IT Energy-Aware Technologies, pp 114–119Google Scholar
  9. 9.
    Jain S, Sritanyaratana S (2003) Method and apparatus to implement the ACPI (advanced configuration and power interface) C3 state in a RDRAM based system, US Patent 6,633,987Google Scholar
  10. 10.
    Wu F (2014) Io-less dirty throttling. In: LinuxCon Japan 2012, LinuxConGoogle Scholar
  11. 11.
    Wilson A (2008) The new and improved filebench. In: Proceedings of 6th USENIX Conference on File and Storage TechnologiesGoogle Scholar
  12. 12.
  13. 13.
  14. 14.
    Manousakis I, Marazakis M, Bilas A (2013) FDIO: A feedback driven controller for minimizing energy in I/O-intensive applications. In: Proceedings of the 5th USENIX Conference on Hot Topics in Storage and File Systems, HotStorage’13, USENIX Association, Berkeley, CA, USA, pp 16–16Google Scholar
  15. 15.
    Schöne R, Hackenberg D, Molka D (2012) Memory performance at reduced CPU clock speeds: an analysis of current x86 64 processors. In: Proceedings of the 2012 USENIX Conference on Power-Aware Computing and Systems, USENIX Association, pp 9–9Google Scholar
  16. 16.
    Chang H-C, Li B, Grove M, Cameron KW (2014) How processor speedups can slow down I/O performance. In: 2014 IEEE 22nd International Symposium on Modelling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), IEEE, pp 395–404Google Scholar
  17. 17.
    Contreras G, Martonosi M (2005) Power prediction for intel xscale® processors using performance monitoring unit events. In: Proceedings of the 2005 International Symposium on Low Power Electronics and Design, 2005. ISLPED’05, IEEE, pp 221–226Google Scholar
  18. 18.
    Economou D, Rivoire S, Kozyrakis C, Ranganathan P (2006) Full-system power analysis and modeling for server environments. In: Workshop on Modeling Benchmarking and Simulation (MOBS), Boston USA, pp 13–23Google Scholar
  19. 19.
    Li T, John LK (2003) Run-time modeling and estimation of operating system power consumption. ACM SIGMETRICS Perform Eval Rev 31(1):160–171CrossRefGoogle Scholar
  20. 20.
    Kim N, Cho J, Seo E (2014) Energy-credit scheduler: an energy-aware virtual machine scheduler for cloud systems. Future Gener Comput Syst 32:128–137CrossRefGoogle Scholar
  21. 21.
    Bertran R, Becerra Y, Carrera D, Beltran V, Gonzàlez M, Martorell X, Navarro N, Torres J, Ayguadé E (2012) Energy accounting for shared virtualized environments under DVFS using PMC-based power models. Future Gener Comput Syst 28(2):457–468CrossRefGoogle Scholar
  22. 22.
    Lewis AW, Ghosh S, Tzeng N-F (2008) Run-time energy consumption estimation based on workload in server systems. HotPower 8:17–21Google Scholar
  23. 23.
    Allalouf M, Arbitman Y, Factor M, Kat RI, Meth K, Naor D (2009) Storage modeling for power estimation. In: Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, SYSTOR ’09, ACM, New York, NY, USA, pp 3:1–3:10. doi: 10.1145/1534530.1534535
  24. 24.
    Prada L, Garcia J, Calderon A, Garcia JD, Carretero J (2013) A novel black-box simulation model methodology for predicting performance and energy consumption in commodity storage devices. Simul Model Pract Theory 34:48–63CrossRefGoogle Scholar
  25. 25.
    Li Y, Long D (2014) Which storage device is the greenest? modeling the energy cost of I/O workloads. In: IEEE 22nd International Symposium on Modelling, Analysis Simulation of Computer and Telecommunication Systems (MASCOTS), pp 100–105. doi: 10.1109/MASCOTS.2014.20
  26. 26.
    Li J, Badam A, Chandra R, Swanson S, Worthington BL, Zhang Q (2014) On the energy overhead of mobile storage systems. In: FAST, pp 105–118Google Scholar
  27. 27.
    Manousakis I, Marazakis M, Bilas A (2013) FDIO: a feedback driven controller for minimizing energy in I/O-intensive applications. In: Presented as Part of the 5th USENIX Workshop on Hot Topics in Storage and File Systems, Berkeley, CAGoogle Scholar
  28. 28.
    Zhu Q, David FM, Devaraj CF, Li Z, Zhou Y, Cao P (2004) Reducing energy consumption of disk storage using power-aware cache management. In: Software, IEE Proceedings, IEEE, pp 118–118Google Scholar
  29. 29.
    El-Sayed N, Schroeder B (2014) To checkpoint or not to checkpoint: understanding energy-performance-I/O tradeoffs in HPC checkpointing. In: IEEE International Conference on Cluster Computing (CLUSTER), IEEE, pp 93–102Google Scholar
  30. 30.
    Kunkel JM, Minartz T, Kuhn M, Ludwig T (2012) Towards an energy-aware scientific I/O interface. Comput Sci Res Dev 27:337–345CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Computer Science and Engineering DepartmentUniversity Carlos III of MadridLeganesSpain

Personalised recommendations