The Journal of Supercomputing

, Volume 71, Issue 2, pp 448–478 | Cite as

Energy-efficient adaptive networked datacenters for the QoS support of real-time applications

  • Nicola Cordeschi
  • Mohammad Shojafar
  • Danilo Amendola
  • Enzo BaccarelliEmail author


In this paper, we develop the optimal minimum-energy scheduler for the adaptive joint allocation of the task sizes, computing rates, communication rates and communication powers in virtualized networked data centers (VNetDCs) that operate under hard per-job delay-constraints. The considered VNetDC platform works at the Middleware layer of the underlying protocol stack. It aims at supporting real-time stream service (such as, for example, the emerging big data stream computing (BDSC) services) by adopting the software-as-a-service (SaaS) computing model. Our objective is the minimization of the overall computing-plus-communication energy consumption. The main new contributions of the paper are the following ones: (i) the computing-plus-communication resources are jointly allotted in an adaptive fashion by accounting in real-time for both the (possibly, unpredictable) time fluctuations of the offered workload and the reconfiguration costs of the considered VNetDC platform; (ii) hard per-job delay-constraints on the overall allowed computing-plus-communication latencies are enforced; and, (iii) to deal with the inherently nonconvex nature of the resulting resource optimization problem, a novel solving approach is developed, that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The sensitivity of the energy consumption of the proposed scheduler on the allowed processing latency, as well as the peak-to-mean ratio (PMR) and the correlation coefficient (i.e., the smoothness) of the offered workload is numerically tested under both synthetically generated and real-world workload traces. Finally, as an index of the attained energy efficiency, we compare the energy consumption of the proposed scheduler with the corresponding ones of some benchmark static, hybrid and sequential schedulers and numerically evaluate the resulting percent energy gaps.


Big data stream computing (BDSC) Virtualized networked data centers Real-time cloud computing Adaptive resource management Energy saving 


  1. 1.
    Cugola G, Magara A (2012) Processing flows of information: from data stream to complex event processing. ACM Comput Surveys (CSUR) 44(3)Google Scholar
  2. 2.
    Baliga J, Ayre RWA, Hinton K, Tucker RS (2011) Green cloud computing: balancing energy in processing. Storage Transp Proc IEEE 99(1):149–167Google Scholar
  3. 3.
    Mishra A, Jain R, Durresi A (2012) Cloud computing: networking and communication challenges. IEEE Commun Mag 50(9):24–25Google Scholar
  4. 4.
    Azodolmolky S, Wieder P, Yahyapour R (2013) Cloud computing networking: challanges and opportunities for innovations. IEEE Commun Mag 51(7):54–62Google Scholar
  5. 5.
    Scheneider S, Hirzel M, Gedik B (2013) Tutorial: stream processing optimizations. ACM DEBS 249–258Google Scholar
  6. 6.
    Lu T, Chen M (2012) Simple and effective dynamic provisioning for power-proportional data centers. Proc CISSGoogle Scholar
  7. 7.
    Rajaraman A, Ullman JD (2011) Mining of massive datasets. Cambridge University Press, Cambridge, p 326Google Scholar
  8. 8.
    Chakravarthy Sh, Jiang Q (2009) Stream data processing: a quality of service perspective, vol 36. Springer, Berlin, p 348Google Scholar
  9. 9.
    Krempl G, Brzezinski D, Hllermeier E, Last M (2014) Open challenges for data stream mining research. ACM SIGKDD Explor NewslettGoogle Scholar
  10. 10.
    Mittal S (2014 ) Power management techniques for data centers: a survey. arXiv:1404.6681
  11. 11.
    Baccarelli E, Biagi M, Pelizzoni C, Cordeschi N (2007) Optimized power allocation for multiantenna systems impaired by multiple access interference and imperfect channel estimation. IEEE Trans Veh Technol 56(5):3089–3105CrossRefMathSciNetGoogle Scholar
  12. 12.
    Neumeyer L, Robbins B, Nair A, Kesari A (2010) S4: distributed stream computing platform. In: International workshop on knowledge discovery using cloud and distributed computing platforms, ICDMW ’10, pp 170–177Google Scholar
  13. 13.
    Zaharia M, Das T, Li H, Shenker S, Stoica I (2012) Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. Hot CloudGoogle Scholar
  14. 14.
    Loesing S, Hentschel M, Kraska T (2012) Storm: an elastic and highly available streaming service in the cloud. EDBT-ICDT ’12, pp 55–60Google Scholar
  15. 15.
    Qian Z, He Y, Su C, Wu Z, Zhu H, Zhang T (2013) TimeStream: reliable stream computation in the cloud. In: EuroSys, pp 1–14Google Scholar
  16. 16.
    Kumbhare A et al (2014) PLAstiCC: predictive look-ahead scheduling for continuous data- flows on clouds. CCGRIDGoogle Scholar
  17. 17.
    Mathew V, Sitaraman R, Rowstrom A (2012) Energy-aware load balancing in content delivery networks. IEEE INFOCOMGoogle Scholar
  18. 18.
    Padala P, You KY, Shin KG, Zhu X, Uysal M, Wang Z, Singhal S, Merchant M (2009) Automatic control of multiple virtualized resources. . In: Proceedings of the 4th ACM European conference on computer systems, pp 13–26Google Scholar
  19. 19.
    Kusic D, Kandasamy N (2008) Power and performance management of virtualized computing environments via look-ahead control. . In: Proceedings of the international conference on automatic computing, vol 1, pp 3–12Google Scholar
  20. 20.
    Govindan S, Choi J, Urgaonkar B, Sasubramanian A, Baldini A (2009) Statistical profiling-based techniques for effective power provisioning in data centers. Proc Euro SystGoogle Scholar
  21. 21.
    Lin M, Wierman A, Andrew L, Thereska E (2011) Dynamic right-sizing for power-proportional data centers. IEEE INFOCOMGoogle Scholar
  22. 22.
    Zhou Z et al (2013) Carbon-aware load balancing for geo-distributed cloud services. IEEE MASCOTS, pp 232–241Google Scholar
  23. 23.
    Tamm O, Hersmeyer C, Rush AM (2010) Eco-sustainable system and network architectures for future transport networks. Bell Labs Tech J 14:311–327CrossRefGoogle Scholar
  24. 24.
    Liu J, Zhao F, Liu X, He W (2009) Challenges towards elastic power management in internet data centers. In: Proceedings on IEEE international conference on distributed computing systems workshops, Los AlamitosGoogle Scholar
  25. 25.
    Khan AN, Mat Kiah ML, Madani SA, Ali M, Khan AR, Shamshirband S (2014) Incremental proxy re-encryption scheme for mobile cloud computing environment. J Supercomput 68(2):624–651Google Scholar
  26. 26.
    Nathuji R, Schwan K (2007) VirtualPower: coordinated power management in virtualized enterprise systems. In: ACM 21th SOSP’07, pp 265–278Google Scholar
  27. 27.
    Kim KH, Beloglazov A, Buyya R (2009) Power-aware provisioning of cloud resources for real-time services. Proc ACM MGC’09Google Scholar
  28. 28.
    Koller R, Verma A, Neogi A (2010) WattApp: an application aware power meter for shared data centers. ICAC’10Google Scholar
  29. 29.
    Warneke D, Kao O (2011) Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Trans Parallel Disturb Syst 22(6):985–997CrossRefGoogle Scholar
  30. 30.
    Zhu D, Melhem R, Childers BR (2003) Scheduling with dynamic voltage/rate adjustment using slack reclamation in multiprocessor real-time systems. IEEE Trans Parllel Distrib Syst 14(7):686–700CrossRefGoogle Scholar
  31. 31.
    Vasudevan V et al (2009) Safe and effective fine-grained TCP retransmissions for datacenter communication. ACM SIGCOMM, pp 303–314Google Scholar
  32. 32.
    Alizadeh M, Greenberg A, Maltz DA (2010) J Padhye “Data center TCP (DCTCP)”, ACM SIGCOMM.Google Scholar
  33. 33.
    Das T, Sivalingam KM (2013) TCP improvements for data center networks. COMSNETS, pp 1–10Google Scholar
  34. 34.
    Kurose JF, Ross KW (2013) Computer networking: a top-down approach featuring the internet, 6th edn. Addison WesleyGoogle Scholar
  35. 35.
    Jin S, Guo L, Matta I, Bestravos A (2003) A spectrum of TCP-friendly window-based congestion control algorithms. IEEE/ACM Trans Netw 11(3):341–355CrossRefGoogle Scholar
  36. 36.
    Baccarelli E, Biagi M, Pelizzoni C, Cordeschi N (2008) Optimal MIMO UWB-IR transceiver for Nakagami-fading and Poisson-arrivals. J Commun 3(1):27–40CrossRefGoogle Scholar
  37. 37.
    Cordeschi N, Patriarca T, Baccarelli E (2012) Stochastic traffic engineering for real-time applications over wireless networks. J Netw Comput Appl 35(2):681–694CrossRefGoogle Scholar
  38. 38.
    Baccarelli E, Cordeschi N, Polli V (2013) Optimal self-adaptive QoS resource management in interference-affected multicast wireless networks. IEEE/ACM Trans Netw 21(6):1750–1759CrossRefGoogle Scholar
  39. 39.
    Al-Fares M, Loukissas A, Vahdat A (2008) A scalable commodity data center network architecture. ACM SIGCOMM, pp 63–74Google Scholar
  40. 40.
    Gulati A, Merchant A, Varman PJ (2010) mClock: handling throughput variability for hypervisor IO scheduling, OSDI’10Google Scholar
  41. 41.
    Ballami H, Costa P, Karagiannis T, Rowstron A (2011) Towards predicable datacenter networks, SIGCOMM ’11Google Scholar
  42. 42.
    Greenberg A et al (2011) VL2: a scalable and flexible data center network. Commun ACM 54(3):95–104Google Scholar
  43. 43.
    Guo C et al (2010) SecondNet: a data center network virtualization architecture with bandwidth guarantees. ACM CoNEXTGoogle Scholar
  44. 44.
    Xia L, Cui Z, Lange J (2012) VNET/P: bridging the cloud and high performance computing through fast overaly networking, HPDC’12Google Scholar
  45. 45.
    Wang L, Zhang F, Aroca JA, Vasilakos AV, Zheng K, Hou C, Li D, Liu Z (2014) Green DCN: a general framework for achieving energy efficiency in data center networks. IEEE JSAC 32(1):4–15Google Scholar
  46. 46.
    Khan AN, Mat Kiah ML, Ali M, Madani SA, Khan AR, Shamshirband S (2014) BSS: block-based sharing scheme for secure data storage services in mobile cloud environment. J Supercomput. doi: 10.1007/s11227-014-1269-8
  47. 47.
    Neely MJ, Modiano E, Rohs CE (2003) Power allocation and routing in multi beam satellites with time-varying channels. IEEE/ACM Trans Netw 19(1):138–152CrossRefGoogle Scholar
  48. 48.
    Wang L, Zhang F, Hou C, Aroca JA, Liu Z (2013) Incorporating rate adaptation into Green networking for future data centers. IEEE NCA, pp 106–109Google Scholar
  49. 49.
    Balter MH (2013) Performance modeling and design of computer systems. Cambridge Press, CambridgeGoogle Scholar
  50. 50.
    Chiang M, Low SH, Calderbank AR, Doyle JC (2007) Layering as optimization decomposition: a mathematical theory of network architectures. Proc IEEE 95(1):255–312CrossRefGoogle Scholar
  51. 51.
    Cordeschi N, Shojafar M, Baccarelli E (2013) Energy-saving self-configuring networked data centers. Comput Netw 57(17):3479–3491CrossRefGoogle Scholar
  52. 52.
    Bazaraa MS, Sherali HD, Shetty CM (2006) Nonlinear programming, 3rd edn. Wiley, New YorkGoogle Scholar
  53. 53.
    Kushner HJ, Yang J (1995) Analysis of adaptive step-size SA algorithms for parameter tracking. IEEE Trans Autom Control 40(8):1403–1410CrossRefzbMATHMathSciNetGoogle Scholar
  54. 54.
    Baccarelli E, Cusani R (1996) Recursive Kalman-type optimal estimation and detection of hidden Markov chains. Signal Process 51(1):55–64CrossRefzbMATHGoogle Scholar
  55. 55.
    Urgaonkar B, Pacifici G, Shenoy P, Spreitzer M, Tantawi A (2007) Analytic modeling of multitier internet applications. ACM Trans Web 1(1)Google Scholar
  56. 56.
    Srikant R (2004) The mathematics of internet congestion control. Birkhauser, BaselGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Nicola Cordeschi
    • 1
  • Mohammad Shojafar
    • 1
  • Danilo Amendola
    • 1
  • Enzo Baccarelli
    • 1
    Email author
  1. 1.Department of Information, Electrical and Telecommunication (DIET) engineering“Sapienza” University of RomeRomeItaly

Personalised recommendations