Skip to main content

The \(CiS^2\): a new metric for performance and energy trade-off in consolidated servers

Abstract

The increased use of cloud services has turned the virtualization in the main technology that supports cloud datacentres. To reduce the increment of power consumption caused in datacenters due to the addition of physical servers, system administrators are using virtual machine consolidation (VMC) techniques which tries to allocate the adequate number of virtual machines per physical server. Therefore, VMC increases server resources utilization and as a consequence its performance degradation and the energy consumption too. Then, a trade-off between the performance and the energy consumption exists when consolidating virtual machines. This trade-off is difficult to quantify and also to determine the servers efficiency taking into account a specific number of allocated virtual machines. Because of this, it is crucial for system administrators having a simple metric that assists the VMC making-decision process. In this paper, we propose the \(CiS^2\) index, a metric to quantify this performance-energy trade-off. Also, this index can help system administrators to decide about the servers’ efficiency through benchmarking and to select the most efficient server through a proposed algorithm. Besides, we propose a simple graphical representation of the index to distinguish graphically the efficient and non-efficient server consolidations. We validate the index in a theoretical manner and performing real experiments in different physical servers under CPU workload saturation. Obtained results show that the proposed index reflects the performance-energy trade-off behaviour and it helps systems’ administrators when consolidating virtual machines.

Introduction

In the last years, organizations started to be concerned about the impact of information technology (IT) in energy consumption. Due to that, they are trying to implement Green IT initiatives to make their businesses environment-friendly, sustainable and cost-effective [36, 38]. Datacentres are becoming an area of IT business where environmental sustainability is imperative especially in cloud companies. In fact, a cloud datacentre comprises thousands of servers offering a variety of services through the Internet including cloud services: SaaS, PaaS, and IaaS [35].

The increasing demand for cloud services has driven building larger datacentres, as massive server farms. The datacentre’s servers consume a huge amount of power and also emit greenhouse gases in the form of CO\(_2\). In a current datacentre, the 30% of servers either are even not used or their utilization ratio is very low, around 5–10% [3, 25]. Also, servers are the most power-demand device of a datacentre as Fig. 1 shows. Also, the IT industry is responsible for about 2% of global CO\(_2\) emissions almost equivalent to the aviation industry [1].

Fig. 1
figure1

Power distribution in a traditional data centre, based on [7]

Moreover, Green IT is used as an umbrella covering overlapping concepts like VMC and power management among many others. Then, the aspiration of Green IT is achieving higher energy efficiency in the use of IT devices and to increase the utilization of already installed devices in datacentres using the virtualization technology [38].

The virtual machine consolidation challenge

Fig. 2
figure2

Virtual machine consolidation, from [7]

A traditional datacentre can be more efficient by using virtualization technologies [31, 37, 38]. In fact, the server consolidation technique is based on the reallocation of virtual machines among different physical servers using virtual machine migration (see Fig. 2). As a consequence, the utilization of physical servers increase and the number of switched-on physical servers can be reduced. The workload of a consolidated server can increase from 50 to 85%, where servers operate more energy efficiently [5, 7]. Therefore, the server consolidation increases the utilization of physical servers. However, due to the possibility of the switch-off some physical servers, the power consumption is reduced. Nevertheless, as [24] states, the energy consumption depends on the overhead inherent to virtualization. The virtualization overhead is the extra workload that the physical server has to perform due to being virtualized, that is, tasks of managing virtual machines and coordinating the access to physical resources [9, 21]. As a consequence, the larger the number of consolidated virtual machines is, the higher the overhead is because of the coordination of simultaneously demand to resources’ access [7].

Therefore, performance degradation occurs due to the overhead of consolidating virtual machines, despite the reduction in power consumption of not using additional servers instead. Consequently, the value of energy consumption depends on both the performance degradation and power consumption [7]. When systems administrators consolidate servers, they are reducing the power consumption of the datacentre since there are physical servers that may be switched off or at least put on standby. Nevertheless, the workload of the physical machine host is then higher, so the performance is further degraded. Therefore, taking into account this feature of VMC, the dual objective is to maximize the performance of the physical machine while minimizing the power and energy consumption. As a consequence, it is necessary to find a trade-off between the energy consumption and performance of the physical servers hosting virtual ones [7].

In certain cases, the energy saving is not compensated with the performance degradation, which will be very high. It could also be the opposite case, that is, high performance of the datacentre may not be able to compensate the energy consumption, and it is necessary to consolidate servers to reduce it [7]. The current challenge in server consolidation is how to determine if a consolidated server is efficient or not. That is, if the performance degradation is compensated by the reduction in power consumption.

Datacentre and server management metrics

In order to manage the energy efficiency and performance degradation of datacentres and servers, systems’ administrators can use metrics as PUE, CUE, % utilization resources, SWaP, and performance per watt, among others. All of these metrics can be calculated measuring datacentre operational values [32].

Due to the trade-off between the power saved and the performance degradation inherent to virtualization technology, systems’ administrators need to determine the effect of VMC in datacentres. The first attempt to measure the energy efficiency and the performance was from the microprocessors point of view, with the energy-delay product (EDP) metric [15]. The EDP metric attempts to compare microprocessors through the ratio of energy and performance under an specific workload.

However, none of the current metrics are able to measure this trade-off in a similar way that EDP but for consolidated virtual machines per physical server, which is the issue that administrators want to optimize [40, 46].

Therefore, it is necessary to have a metric which can relate the three issues of server consolidation simultaneously:

  • Performance degradation of the physical server in terms of consolidated virtual machines.

  • Energy consumed by the physical servers with and without this performance degradation due to consolidation.

  • The ability to compare objectively the physical server in the above conditions against other physical servers (benchmarking).

Contribution

In this paper, we present a new metric, named Consolidated index for CPU-Server Saturation (\(CiS^2\)), which attempts to express the performance-energy trade-off when consolidating virtual machines. In previous work [6], we started to introduce an overview of this metric and the first experimental observations. However, in this work, we depict a more detailed description of it and an extended experimentation. Specifically, we select the most efficient server in terms of performance and energy efficiency for virtual machine consolidation among a set of different servers. This selection process is shown in Fig. 3 and its main advantage is the non-necessity of having a reference machine for benchmarking the set of servers. That is, our contribution attempts to compare a physical server with itself when consolidating virtual machines, and, also to compare this sever with others in the same experimental conditions. We will detailed these two comparisons in next sections and in Table 6.

The problem statement and motivation are depicted in Sect. 2. In Sect. 3 we depict the background concepts related to performance and energy metrics in datacentres and servers. Then, in Sect. 4 we introduce concepts related to performance, power and energy consumption, the efficiency of datacentres, which are required to understand the proposed metric. In Sect. 5 we expose our contribution, the \(CiS^2\) as a new metric of the performance-energy trade-off for consolidated servers. In addition, in Sect. 6 we propose a process for benchmarking and select the most efficient server through the \(CiS^2\) values. Then, in Sect. 7, we evaluate the \(CiS^2\) from a theoretical and a practical point of view. Furthermore, in Sect. 8, we propose a graphical and simpler analogy to understand the \(CiS^2\) obtained from the server measurements. To finish, in Sect. 9 we conclude and propose future research work lines.

Fig. 3
figure3

Servers’ comparison process

Motivation and problem statement

Multi-tenant datacentres provide services to customers on a lease basis. They also provide space and/or services to an individual enterprise that place and manage their own equipment while the provider manages the cooling and facility infrastructure [45].

The virtualization technology allows the multi-tenancy environment, which has several impacts in system performance and energy efficiency. Due to the huge increment in the use of IaaS cloud services, the virtualization technology is gaining huge importance, mainly the VMC technique [33]. For this reason, it is important to assess and ensure the efficiency of virtualized datacentres is terms of performance and energy. Besides, the datacentre measurement is one of the Green IT principles [17], providing systems’ administrators the required information to decide about the efficiency of the VMC.

In this work, our concern is to determine the efficiency of a server through the performance-energy trade-off when consolidating. That is, to quantify the impact of the VMC in the physical server that supports the virtualization, as Fig. 4 shows.

Fig. 4
figure4

Problem scope

Therefore, the research question we attempt to solve in this work is:

  • RQ: How can we express the performance-energy trade-off of physical servers when consolidating virtual machines?

As the Green IT principles state [17], the consolidation is one way to improve the datacentre efficiency, but, it is necessary to measure the datacentre and servers’ state. In this case, the servers’ needs to take into account the number of virtual machines are consolidated in order to assess its state.

As the next section shows, there is a large set of metrics for assessing the datacentre and servers’ efficiency. However, the current metrics do not allow us to assess the datacentre from a VMC trade-off perspective.

Background in datacentre and server metrics

Since the energy efficiency and performance degradation are the main concerns of systems’ administrators, many research works have been developed during last years.

Nevertheless, in this work we are interested in the server consolidation point of view, since the microprocessors’ energy efficiency measurement concern is currently solved through EDP metric. These works attempted to tackle the management of these issues proposing several metrics to quantify the performance and the efficiency of a datacentre and servers is in terms of consumed energy and power, as well as performance-energy trade-off have been proposed.

The main developed work [14] during last years explores the diverse metrics that are currently available to measure numerous datacentre infrastructure components behaviour. Also, they proposed a taxonomy of metrics based on datacentre dimensions. In addition, the authors argue for the design of new metrics considering factors such as locations and resource co-locations, in order to assist in the strategic datacentre design and operations processes. One of the challenges authors announced is that it is hard to know the energy consumption due to datacentre sub-components, as operating systems and virtual machines. Due to that, in this work, we propose a metric for performance-energy trade-off, which takes into account the number of allocated virtual machines.

On the one hand, the performance metrics attempt to quantify the suitability of the amount of work accomplished by a server or a datacentre. The values can be directly monitored from the system or inferred them. In the same manner, the energy and power metrics quantify the consumption of power or energy of a datacentre and/or a physical server. In order to take into account both previous aspects simultaneously, it is necessary to measure the relationship between performance and energy. These metrics relate the performance of servers or datacentres with the power or energy consumption.

Fig. 5
figure5

Power management techniques at different levels in datacenters, as appears on [22]

Power management metrics and techniques at different levels in datacenters are shown in Fig. 5. Systems’ administrators may measure information from software and hardware optimization. In this work, we are focused on metrics regarding software-oriented optimizations, specifically VMC, and, hardware-oriented optimizations, focused on the power and energy consumption reduction in physical servers.

The core dimensions of datacentre operations are energy efficiency, cooling, greenness, performance thermal and air management, network, security, storage and financial impact [14]. However, we take into account subcategories related to computing tasks, as performance, power/energy consumption and performance-energy trade-off, summarizing the most relevant metrics in Table 1.

Moreover, a taxonomy based on the overview of the datacentre energy consumption is shown in Fig. 6. The behaviour of the energy consumption can be evaluated and predicted from software and hardware point of view, taking into account different elements from each category. In this work, we are focused on the virtualization technology and the CPU-intensive applications, from the software point of view. Besides, from the hardware point of view, we are focused on the energy-efficiency of the computation components, as the processor unit.

Performance metrics

The state-of-the-art shows that we can extract performance metrics from single servers and the datacentre as a whole. In both cases, the aim is to know the system’s behaviour. Also, metrics should be aligned with the abstraction level. From the server point of view, it is possible to measure the amount of CPU utilization and the throughput of the datacentre as processed requests per time from the datacentre.

Since we want to establish a server metric taking into account the consolidated virtual machines, the workload will be in function of the number of consolidated virtual machines. Due to that, there is necessary to take into account other performance metrics as performance degradation due to consolidation.

Energy and power efficiency metrics

The aim of efficiency metrics is to quantify how much efficient a system is (a datacentre or a single server) under the execution of a specific workload, in terms of power and/or energy. In the same manner, system administrators can measure the efficiency and the power/energy consumption from a different level of abstraction. From the server point of view, they can measure the power consumption or the energy. Besides, they can measure the PUE at datacentre. As the previous case, metrics should be aligned with the abstraction level. In this work, we are interested in having a metric which relates the energy and power consumption with the number of consolidated virtual machines, from the server point of view as a single black box for computation.

Performance and energy trade-off metrics

The trade-off metrics express the relation between the amount of system performance and the energy or power consumed by a system executing a specific workload. The performance metrics are related to the executed workload depending on the system’s resources and their utilization, in our case, the percent of CPU. However, we are interested in consolidating virtual machines [27, 29]. We have to take into account how much power consumption is saved against the performance degradation. As a consequence, our attempt is to quantify the performance-energy trade-off of a system taking into account the number of consolidated virtual machines.

Despite the set of metrics presented in Table 1, none of them are designed for comparing datacentres or server amongst each other. Although the PUE metric is currently used for this purpose, it was never intended to be used as a comparison metric [32]. The PUE is also influenced by the location of the datacentre. Therefore, comparisons between datacentres using PUE are most often not representative of the current situation (virtualized datacentres). Moreover, metrics such as EUE (cpu), SPPE and SPPP quantify the performance-energy trade-off when consolidating. However, these metrics are not able to depict the contribution of performance degradation and power consumption separately in consolidation.

Since we aim to compare a server when consolidating and to compare a server among a set of other servers, to the best of our knowledge, older facilities may not be able to capture the raw data that feeds today’s more sophisticated metrics. There is no metric to quantify the performance-energy trade-off taking into account the number of consolidated virtual machines in a physical server [32]. For this reason, in this paper, we present the \(CiS^2\) metric which attempts to assist the VMC assessment process.

Fig. 6
figure6

A taxonomy based on the overview of the datacenter energy consumption modelling [12]

Table 1 Current main performance and energy efficiency servers and datacentre metrics

Performance and energy consumption in servers

In order to show the usefulness of the \(CiS^2\) index, it is necessary to define some concepts and the scope of our research focused on performance, energy consumption and efficiency of consolidated servers.

Performance

The term performance is defined as the amount of work accomplished by a system [26]. The suitability of the work accomplishment can be measured meanly with the response time (R), which is the total amount of time that the system takes to respond to a service request, i.e., the time that virtual machine takes to execute a specific workload.

Frequently, it is interesting to define the system performance as the inverse of the response time. That is, the less response time, the more performance. Let’s consider two systems X and Y (e.g. physical and consolidated server), which response time are \(T_X\) and \(T_Y\) respectively. If \(T_X=T_Y\) the performance of two systems is equal or equivalent, since both response times take the same amount of time to execute a workload. On the contrary, if \(T_X<T_Y\), that is, the response time of system X is less than the Y one. Under these conditions, we can assert that “system X is faster than system Y”. Nevertheless, to quantify this relation, the term speedup (SP) is used: “system X is x times faster than system Y” (see Eq. 1).

$$\begin{aligned} SP = \frac{T_X}{T_Y} \end{aligned}$$
(1)

Therefore, the speedup represents the system performance increment (or decrement) respect to another. For the VMC case, the response time for the consolidated server will decrease due to the virtualization overhead, implying a system slowdown.

In addition, it is important to highlight that a system resource is any physical or virtual component of limited availability within a computer system. In this paper, we are focused on the physical CPU (or the set of physical CPUs). Specifically, the CPU will be saturated, that is, all the physical CPUs will be near 100% of utilization. With this assumption, we are evaluating the system in the worst CPU state, from the performance point of view [11].

Power, energy consumption, power and energy efficiency

Power consumption refers to the consumed electricity, supplied to operate servers or a datacentre. It is usually measured in units of watts (W) or kilowatts (kW). In a server environment, the CPU is the most power-demanding device (see Fig. 7) [25], defining the power consumption in function of its utilization [4].

Fig. 7
figure7

Server power consumption, from [25]

In addition, energy consumption is defined as the sum of power consumption over a period of time (see Eq. 2). Also, the energy efficiency is defined as the amount of work per energy consumption over a time period (T), and power efficiency as the work rate per power usage [1].

$$\begin{aligned} energy= \int _{0}^{T} power_i \end{aligned}$$
(2)

Performance and energy speed-up in server consolidation

Before determining the \(CiS^2\), it is important to define some additional metrics taking into account the speed-up defined in Eq. 1. For an intensive CPU workload balanced-distributed, let’s define \(\overline{R^C}\) as the mean response time for a consolidated server, with N identical virtual machines, and \(\overline{R^{PM}}\) as the mean response time of the physical server. We define the speed-up of performance as the ratio of \(\overline{R^C}\) and \(\overline{R^{PM}}\) (see Eq. 3). If \(SP_p > 1\), the average response time of the consolidated server is greater than the average response time of the physical server. As a consequence, the consolidated server has less performance than the physical one.

$$\begin{aligned} SP_p = \frac{\overline{R^C}}{\overline{R^{PM}}} \end{aligned}$$
(3)

In the same manner, let’s define \(\overline{E^C}\) as the average energy consumption of the consolidated server, hosting N identical virtual machines, and \(\overline{E^{PM}}\) as the average energy consumption of the server. We define the energy speed-up as the ratio of \(\overline{E^C}\) as the average energy consumption of the consolidated server and \(\overline{E^{PM}}\) (see Eq. 4). If \(SP_e > 1\), the average energy consumption of the consolidated server is greater than the average energy consumption of the physical server. Therefore, the consolidated server consumes more energy consumption than the physical one.

$$\begin{aligned} SP_e = \frac{\overline{E^C}}{\overline{E^{PM}}} \end{aligned}$$
(4)

System efficiency

In this case, the system efficiency is defined as the pair: performance speed-up and energy speed-up \((SP_p,SP_e)\). Taking into account Eqs. 3 and 4, two rules of thumb of efficiency should be considered (see Fig. 8).

On one hand, we may consider that “the performance degradation of N virtual machines consolidated in the same physical machine should be linear”, that is equivalent to say “the speedup of having N homogeneous parallel physical servers sharing the workload should be equivalent to N times the performance (degradation) of having N virtual machines in just one physical machine sharing the same workload”. We present this linear case as \(SP_p=N\).

On the other hand, since our concern is the power consumption of a whole server either set of servers, we may consider that “the amount of energy consumed for a number of consolidated virtual machines allocated in the same physical server should be similar to the corresponding same number of identical physical machines”. This should be equivalent to say “the increment of power consumption of having N identical physical machines should be compensated with the increment of response time of having N virtual machines in consolidation in just one of these physical machines”. Then, the energy should be equivalent. We represent this ideal case as \(SP_e=1\).

Fig. 8
figure8

Graphical representation for the rules of thumb

The consolidation index for CPU workload in server saturation (\(CiS^2\))

A metric is generally defined as the empirical, objective assignment of numbers, according to a rule derived from a model or theory, to attributes of objects or events to describe that model or theory [14]. The metrics we have included in the related work do not reflect the relationship between the performance and energy consumption of a consolidated server. For this reason, in this section, we present a new metric: Consolidated index for CPU-Server Saturation (\(CiS^2\)) which attempts to quantify the performance and energy trade-off taking into account the number of consolidated virtual machines per physical server.

Also, other research works show that the power consumption and energy depend on the utilization levels of active components [42]. Based on this, the \(CiS^2\) metric should be taking into account the following features:

  • The resources that consume more power should be more important, that is, the CPU should get more weight as it is the largest power consumer component [4, 25]. Also, the more utilization of CPU, the more power consumption. Then, this work is focused on the CPU on its maximum utilization.

  • The physical server performance in a virtualized environment depends directly on the number of consolidated virtual machines. That is, the more allocated virtual machines, the more performance degradation [3].

  • The energy consumption in a virtualized environment depends directly on the performance degradation and the nominal power of the physical server hosting virtual machines.

As we state previously, there is no metric that satisfies the previously required features for a performance-energy trade-off metric in VMC. Moreover, any system administrator would desire to have a metric that allows them to distinguish different consolidation configurations. In addition, a system administrator should be able to select those scenarios which benefit performance and energy efficiency at the same time. Another interesting feature of the index would be to easily distinguish (in a graphical manner) those desirable cases from other less desirable. Thus, metric quantities, their graphical representation and rules of thumb approximations about performance and energy should be consistent. In this section, we aim to fill this research gap [10].

Definition

The Consolidated index for CPU-Server Saturation (\(CiS^2\)) is defined as the product of the speed-up of performance and the speed-up of the consumed energy (see Eq. 5). Therefore it is a simple quadratic efficiency and also the name of \(CiS^2\). Also, in Fig. 9 we present the desirable index in function of the possible number of consolidated virtual machines. In the vertical axis, we represent the combined performance and energy efficiency values. In the horizontal axis the number of machines to be consolidated, either they are available in the datacentre or they are only considered for future capacity planning and forecasting bottlenecks.

$$\begin{aligned} CiS^2 = SP_p \cdot SP_e \end{aligned}$$
(5)
Fig. 9
figure9

\(CiS^2\) index values and reference diagonal

Graphical representation and interpretation

The representation for the \(CiS^2\) index values, with respect the incremental number of machines, would be a square where the reference diagonal separates the scenario configurations of high performance and low energy from those that degrade performance and/or consume excessive energy in relation with the following rules of thumb: “N virtual machines consolidated in one physical machine should be N times slower than N physical machines (parallel replicas)”, and, “N virtual machines in one physical machine should consume as energy as N physical machines (since N virtual machines mean response time should compensate the energy of the linear increment of power of N physical machines)”.

On one hand, the more consolidation of virtual machines hosted in physical servers, the more performance degradation and consequently, the energy consumption would increase due to the increment of the mean response time. On the other hand, the more physical machines used, the more power is consumed, and consequently, the energy is also increased. Thus, we argue that it is possible to measure the balance between both situations. Due to that, the \(CiS^2\) index compares different configurations in the performance-energy trade-off between different VMC scenarios.

From the energy efficiency point of view, the balanced efficiency metric shown through \(CiS^2\) should be the one in which the average energy of a number of consolidated physical machines in a number of virtual machines is exactly the same of using corresponding physical machines, i.e. the energy speed-up is equal to one (\(SP_e=1\)). However, from the point of view of performance speed-up, the balanced efficiency shown through \(CiS^2\) should be the one in which performance degradation is linear, that is, the slowdown is exactly the same of the number of consolidated virtual machines per physical machine, i.e. N is the number of machines(\(SP_p=N\)).

Desirable values

Being \(CiS^2\) values the result of the product of performance and energy speed-up, \(CiS^2\) index acts as a qualifier of the consolidation of a number of virtual machines in comparison with this balanced (and pessimistic) reference diagonal described by the application of the rules of thumb. Therefore, we also define the \(CiS^2\) reference diagonal as the imaginary border separating the “inefficient” \(CiS^2\) values (above the line) from the “efficient” \(CiS^2\) values (below the line) as we depicted in Fig. 10. This reference diagonal represents the linearity of the consolidation in terms of performance and energy, so that increasing consolidation would mean lowering proportionally the performance and also it means exchanging power by energy [6].

Fig. 10
figure10

Examples of values for the \(CiS^2\) index

Having two different areas to distinguish among different consolidations for one server or to compare different servers’ consolidations depending on the area position of values, is another interesting feature of the \(CiS^2\) index for system administrators. Any consolidation configuration is more efficient or more inefficient, depending on the Euclidean distance of the index to the reference diagonal, above or below the reference diagonal, respectively as it is shown in Fig. 10a.

In order to illustrate the practical use of the reference diagonal for system administrators decisions about configurations, we provide 4 different examples shown in Fig. 10b.

On the one hand, if the \(CiS^2\) value is contained in the inefficient area, the inefficiency of this value increases when the Euclidean distance between the value and the reference diagonal increases. For example, in Fig. 10b there are some examples of representative values as dots. For the inefficient area, we may establish that the dot # 4 is more inefficient than the dot # 3 because the Euclidean distance between the reference diagonal and the value represented by # 4 is higher than the Euclidean distance between the reference diagonal and the dot # 3. On the other hand, if the \(CiS^2\) value is contained in the efficiency area, the efficiency of this value increases when the Euclidean distance between the value and the reference diagonal increases. In the same manner, in Fig. 10b we can see representative dots in the efficient area. The dot # 2 is more efficient than the dot # 1 because the Euclidean distance between the reference diagonal and the value represented by the dot # 2 is higher than the Euclidean distance between the reference diagonal and the dot # 1.

The use of the \(CiS^2\): server benchmarking in consolidation

In the previous sections, we defined the \(CiS^2\) metric, as well as its origins and its necessity. In addition, we highlighted that this metric allows a system administrator to distinguish the degree of efficient consolidation of a server compared to not using virtualization in a quick look that particular \(CiS^2\) value and the Euclidean distance to the reference diagonal. However, it is possible to formalize the \(CiS^2\) values, not only to compare different consolidation configurations but also make benchmarking between physical servers.

Let’s define S as the physical server set, then, given a physical server \(s \in S\) its values of \(CiS^2\) (vertical axis) are calculated from the multiplication of the performance efficiency and the energy efficiency of executing the ith uniformly distributed fraction of certain benchmark in homogenous consolidated virtual machines (horizontal axis) versus executing the same workload fraction in parallel replicas of each \(s \in S\), respectively (see Eqs. 3, 4 and 5, and Fig. 11).

For any server \(s \in S\), let’s define the scalability vector \([(1, CiS^2_1)... (n, CiS^2_i)]\), which has n positions for each number of i machines, where \(1\le i \le n\), executing the benchmark (virtual vs. physical) [18]. Thus, we define:

  • The \((i, CiS^2_i)\) value represents the quadratic efficiency described in Eq. 5 for a number of i homogenous virtual machines consolidated in \(s \in S\) vs. i replicas of physical server \(s \in S\).

  • The number of scalability vector components (all the \((i,CiS^{2}_i)\) values for the server s) is limited by the number of machines, i.e. the different scenarios for incremental consolidation of machines \(s \in S\).

  • The size of the scalability vector is n, corresponding to the maximum number of homogeneous virtual machines consolidated in \(s \in S\) executing in parallel the uniformly distributed \(n\)th fraction of the benchmark. That is, for any server \(s \in S\) it is impossible to use more than n virtual machines executing the benchmark because there is a lack of resources (virtual or physical) or there are other restrictions impeding consolidating more virtual machines per server (for example the sample size selected for the different scenario comparisons).

  • Within the scalability vector for server \(s \in S\) (all the \((i, CiS^2_i)_n\)) values, there is one \((h, CiS^2_h)\) that corresponds to the highest \(CiS^2\) value in the vector, when consolidating h machines in a server \(s \in S\) vs. h physical replicas of \(s \in S\). We named this point, \((h, CiS^2_h)\), as the inflection point and h the number of machines at the inflection point.

Therefore, given a server \(s \in S\), which consolidates up to n homogeneous virtual machines executing a benchmark uniformly distributed among them vs. having up to n physical replicas of \(s \in S\). We establish, for any set of m servers with different physical resources and different physical features and/or configurations, that it is possible to compare their performance and energy trade-off by comparing their corresponding \(CiS^2\) values, scalability vector sizes, inflection point and the number of machines at the inflection point, as follows:

  • Given m servers with scalability vectors \(V = [(1, CiS^2_1)... (n_1, CiS^2_{n_1})]... [(1, CiS^2_1), ... ,( n_m, CiS^2_{n_m})]\), the most efficient \(s_k\) is the \(s_k\) server has the scalability vector with the largest size AND the lowest \(CiS^2_{h_k}\) of the \((h_k, CiS^2_{h_k})\) inflection point.

  • Given m servers with scalability vectors \(V = [(1, CiS^2_1)... (n_1, CiS^2_{n_1})] ... [(1, CiS^2_1), ... ,( n_m, CiS^2_{n_m})]\), where two or more servers \(s_k,...,s_l \in S\) have similar \(CiS^2_{h_k},...,CiS^2_{h_l}\) for their inflection points, the most efficient server \(s_k\) is the one with the highest value of \(n_k\), i.e. the highest size of the scalability vector.

  • Given m servers with scalability vectors \(V = [(1, CiS^2_1)... (n_1, CiS^2_{n_1})] ... [(1, CiS^2_1), ... ,( n_m, CiS^2_{n_m})]\), where two or more servers \(s_k,...,s_l \in S\) have \(n_k,..., n_l\) identical scalability vector sizes, the most efficient server \(s_k\) is the one with the lowest \(CiS^2_{h_k}\) of the \((h_k, CiS^2_{h_k})\) inflection point.

  • Given m servers with scalability vectors \(V = [(1, CiS^2_1)... (n_1, CiS^2_{n_1})] ... [(1, CiS^2_1), ... ,( n_m, CiS^2_{n_m})]\), where two or more servers \(s_k,...,s_l \in S\) have \(n_k,..., n_l\) identical scalability vector sizes AND similar \(CiS^2_{h_k} ,..., CiS^2_{h_l}\) for their inflection points, the system administrator should select the most efficient server \(s_k\) depending on the value of \(h_k\), i.e. the number of machines at the inflection point he/she prefers to deploy.

  • Given m servers with scalability vectors \(V = [(1, CiS^2_1)... (n_1, CiS^2_{n_1})] ... [(1, CiS^2_1), ... ,( n_m, CiS^2_{n_m})]\), where two or more servers \(s_k,...,s_l \in S\) have \(n_k,..., n_l\) identical scalability vector sizes AND similar \(CiS^2_{h_k},..., CiS^2_{h_l}\) for their inflection points AND have \(h_k,..., h_l\) identical number of machines at inflection point, therefore all these servers \(s_k,...,s_l \in S\) are analogous for the performance and energy trade-off when consolidating virtual machines.

Thus, the use of the reference diagonal and the Euclidean distance to the reference diagonal give to the system administrators sufficient information for a given server configuration and even to compare several of them. Also, the scalability vector size, the inflection point and the number of machines at the inflection point characterize the benchmarking for the comparison between servers and their consolidation scenarios.

From the properties exposed before, we can obtain a server-selection algorithm in function of scalability vector values (see Algorithm 1). As we stated before, the algorithm requires as an input the scalability vector for each server \(s_m \in S\) and it generated a list of servers \(s \in S\) as output. Depending on the vector values, the algorithm selects the most efficient servers. As we can see in the algorithm, there is a case which needs the system management support due to the server’s equivalence, in terms of efficiency.

Fig. 11
figure11

Workload execution comparison

figurec

Due to the several factors influencing the performance of virtualization [17] such as the workload type, the number of CPUs and the amount of main memory, we will select different servers and three CPU workload types. The selected workloads are the Sysbench benchmark (prime number calculation) and two workloads from SPEC CPU2017 benchmark (float point and compilation operation). These workloads have been executed in a Dell Power Edge T430 server in CPU-saturation conditions (see Table 2 and Fig. 12). Also, we will execute the Sysbench workload in the Dell Power Edge T430 and Power Edge T330 servers (see Table 3 and Fig. 13).

Computing the \(CiS^2\) value for each workload, we obtain the values depicted in Table 2 and Fig. 12. We can see that for each type of workload, there is no distinction between the scalability values, then, we need to analyse the \(CiS^2\) values at the inflection point to determine which function is more efficient. For this case, the minimum \(CiS^2\) value at the inflection point corresponds to the Sysbench workload. As a result, the most efficient workload for the Power Edge 430 server consolidation is the Sysbench.

For the comparison between the PowerEdge T430 server and the PowerEdge T330 one was done executing the Sysbench workload in the same conditions. It is important to note that the difference between these two servers is just the RAM amount, having 8 GB and 16 GB respectively. The \(CiS^2\) function for each server is depicted in Fig. 13, and, applying the proposed algorithm we could select the T330 server as the most efficient because the \(CiS^2\) value at the inflection point is the minimum (see Table 3).

Table 2 Scalability values for different CPU-intensive workloads for the T430 server
Table 3 Scalability Sysbench-values for server selection
Fig. 12
figure12

\(CiS^2\) function for different CPU-intensive workload in T430 server

Fig. 13
figure13

\(CiS^2\) function for Sysbench workload in T430 and T330 server

\(CiS^2\) evaluation

In previous sections, we proposed a new index (the \(CiS^2\)) to quantify the performance-energy trade-off in function of the number of consolidated machines. This section aims to evaluate the suitability of the index, in a theoretical and practical manner.

Theoretical evaluation

In order to evaluate the \(CiS^2\) index in a theoretical manner we proceed taking into account the index-requirement definitions from [1, 23] and then, to apply them to the \(CiS^2\) index.

  • Quantifiability: if performance and energy metrics are not quantitative by nature, they have to be transformed. In this case, the value of the \(CiS^2\) is a real number resulting of multiplying the performance measured in seconds and the energy measured in watts \(\cdot \) seconds.

  • Sensitivity: it expresses how much the performance and energy should change before the change can be detected. Therefore, a sensitive metric is able to detect even minor changes in performance or energy. The \(CiS^2\) value depends on the precision of the underlying measurements. Also, the \(CiS^2\) metric is very sensitive even for minor changes on the server performance or energy, assuming that a change is a variation in the number of hosted virtual machines.

  • Linearity: it indicates the extent to which process performance or energy changes are congruent with the value of a certain metric. Or, conversely, a small change in the value of a corresponding performance and energy metric, whereas an ample performance and energy rise should also lead to a strong change in the level of the performance and energy metric. The \(CiS^2\) is calculated as the product of efficiency of performance (seconds) and the efficiency of the energy consumption (watts \(\cdot \) seconds). Then, the time has more weight in the \(CiS^2\) value and a small change in this value modifies the \(CiS^2\) quadratically.

  • Reliability: a reliable performance and energy metric is free of measurement errors. The value of \(CiS^2\) metric is computed using measurable facts and there are no subjective components included in the measurements.

  • Efficiency: since the measurement itself requires human, financial and physical resources it must be worth the effort from a cost/benefits point of view. In this case, computing the \(CiS^2\) value is lightweight and requires little computing power. The response time can directly read from workload monitors from OS (software) while the power consumption can be directly read either from the power monitor (hardware) or from software monitors included in servers. Moreover, the calculation of the energy consumption as the product of mean response time and the average power consumption is not a weight computing process.

  • Improvement-oriented: performance and energy metrics should emphasize improvements. The \(CiS^2\) metric measures the performance and energy of the process, neither errors or non-optimal behaviour of the personnel operating a datacentre.

Empirical evaluation

From a practical point of view, we evaluate the proposed index through the study of the \(CiS^2\) index values of several servers, varying the number of consolidated virtual machines. This allows us to study how the different number of consolidated virtual machines in a server affects its performance-energy trade-off [8].

The experiment methodology is based on the comparison of a CPU-intensive workload [30, 43]. Besides, the workload is distributed in a balanced manner over N physical machines against the same workload distributed over N virtual machines consolidated in a single physical server. The comparison is done in terms of both performance efficiency and energy efficiency.

As a consequence, we need to select the more representative CPU-intensive workload from the available ones, as [11] state previously. In the same manner, we select the Sysbench CPU and Stress-ng workload as a representative CPU-intensive workload.

Both workloads allow testing the CPU under different levels of utilization and workload complexity. On the one hand, Sysbench CPU verifies prime numbers by doing standard division of the input number by all numbers between 2 and the square root of the input number. It also allows specifying the number of workers (or physical CPUs) as input. On the other hand, stress-ng multiplies square matrix and allows to set as input the size of the matrix and number of workers (or physical CPUs) that do the multiplications [11].

In this work, we take the system under test (SUT) as a black box, applying benchmarking and monitoring as performance engineering techniques [26]. The SUT executed a benchmark workload and its performance and power consumption are monitored during the execution. We selected the response time as the performance metric.

We compared the workload execution between the physical server and the virtual machines set when the physical CPU is 100% utilized. That is, we evaluate the \(CiS^2\) for the set of servers in the worst power conditions and several levels of complexity CPU-intensive workloads. Therefore, we illustrate the usefulness of the proposed index for SPEC CPU2006 workload, when the CPU is not working at 100% of utilization. Therefore, varying the number of virtual machines hosted in the physical server, the whole workload is distributed between the set of virtual machines (see Fig. 11).

To execute the explained experiment we use physical machines from our lab and the Future SOC Lab infrastructure from the Hasso-Plattner Institut. Also, we design an architecture composed of four entities, which are the following:

  • SUT: we executed the experiment in the different physical machines (see Table 4).

  • Power meter: we measured the power consumption of the SUT with Chroma 66200 device. However, for the Future SOC Lab infrastructure, we measured the power consumption with a software meter.

  • Virtual Machine Monitor: Kernel-Based Virtualization (KVM).

  • Virtual Machines: Ubuntu Server 16.04, 1 GB RAM and the same number of physical cores, in any case.

Table 4 Server features in performed experiments

Statistical assurance

Since the evaluation of the \(CiS^2\) index will be performed from two perspectives, the theoretical and the practical one, it is important to assure the statistical significance of the practical evaluation [20, 26].

In Table 5, we show the mean response time and the standard deviation from the physical and the consolidated execution, for each type of server. It can be noted that for any mean response time value, the deviation from the average value is always less than 5%. That is, the experiments were performed a specific amount of times that assures the statistical significance.

As a conclusion, we can say that the results obtained from real experimentation are suitable to prove the \(CiS^2\) properties.

Table 5 Mean response time and standard deviation for each physical machine type (physical and consolidated execution)

Observed phenomena

In Fig. 14, the mean response time of the physical machines is represented (normalized) in function of the number of distributed machines. We can observe that for any kind of physical server, the response time has the same behaviour. That is, when the number of machines increases, the workload for each machine is lesser, then, the response time decreases. Therefore, the behaviour of the response time does not depend on the hardware features of the physical CPUs.

Regarding the consolidated server, the mean response time (normalized) is represented in Fig. 15, which is in function of the number of distributed machines. We observe that the more machines, the less response time. However, there is not a general behaviour for any physical server. Then, when we consolidate virtual machines in a physical server, the response time depends on the hardware features, the virtual machine administrator scheduler and the workload’s nature.

As we explained before, we measured the mean response time and the average power consumption of the SUT. Then, we can calculate the average energy consumption as the sum of the power consumption values among the response time. In order to increase the readability, we measured the energy consumption in watts \(\cdot \) second. In Fig. 16, the average energy consumption (normalized) of the physical servers is represented. When the number of machines increases, the average energy consumption decreases, due to the response time. Also, for any kind of physical server, the average energy consumption goes down among the number of machines.

On the other hand, the average energy consumption for a consolidated server has not a general behaviour. We can see in Fig. 17 that the average energy consumption (normalized) depends on the number of consolidated machines.

Fig. 14
figure14

Mean response time of physical servers

Fig. 15
figure15

Mean response time of consolidated servers

Fig. 16
figure16

Average energy consumption of physical servers

Fig. 17
figure17

Average energy consumption of consolidated servers

\(CiS^2\) index computation

In Fig. 18, we can see the \(CiS^2\) values for each type of server. The first we can observe is that the \(CiS^2\) values have the same shape, that is, it starts increment and then it goes down when the physical machine has a certain number of allocated virtual machines. Therefore, there is an inflection point which determined the number of minimum consolidated virtual machines the server needs in order to have a good \(CiS^2\) value. The inflection point depends on the physical server, and, as a consequence, it depends on the physical resources the server has. For the Sysbench CPU workload in CPU server saturation, the inflection points are shown in Table 6.

Moreover, in Fig. 19 we can see the \(CiS^2\) index for the Sysbench workload in comparison with the Stress-ng workload, for the Dell Power Edge T430 server. We can observe that for different nature in CPU workload (prime number and matrix operations), the behaviour of the \(CiS^2\) index is the same. It starts growing, and it goes down after the inflexion point. Also, the inflexion point is the same for both workloads.

Therefore, as Table 6 shows, the more physical CPUs, the less virtual machines are needed as the number of machines at inflection point. The powerful server in terms of hardware is the RX600S5-1 one, and, it has the best \(CiS^2\) value.

Moreover, if the hardware difference are not so significant, as the hardware of the Power Edge T330 and the Power Edge T430, the \(CiS^2\) values are similar in consolidation, and also, the inflexion point has the same number of virtual machines. Besides, the R310 server has lesser hardware quality than the previous ones, but, the inflexion point and the scalability vector is the same despite of having less \(CiS^2\) value. The worst hardware case is from the T3400 server having less number of virtual machines at the inflexion point and the scalability vector.

Therefore, the hardware is a determinant factor in the \(CiS^2\) value. Then, we obtain a profile of physical machines, characterized by the \(CiS^2\) index and the number of optimal consolidated virtual machines. Also, we can compare the set of servers among them using the \(CiS^2\) index.

Fig. 18
figure18

\(CiS^2\) values for used servers

Fig. 19
figure19

\(CiS^2\) values for T430 server: Sysbench CPU vs Stress-ng

Table 6 Server selection depending of the parameters values obtained from benchmarking (sorted by server efficiency)

The poolside deckchair analogy

To add another example in a more casual but really illustrative way, let’s use a quite graphic analogy: imagine a poolside deckchair like the one in Fig. 20. Let’s assume that we want to establish a comfortability index of the deckchair to be reclining, in a similar way we have the \(CiS^2\) index. We could establish the rule of thumb that “the more vertical the backrest is, the more uncomfortable to be lying down; therefore, the more horizontal the backrest is, the more comfortable to lie down”. On the other hand, we could consider another rule of thumb: “the more positions of lace have the backrest, the more can the comfort be adjusted, and therefore, the fewer positions of the lace, the less comfort adjustment is expected”. Moreover, let’s guess that the reference of the backrest inclination for being comfortable is 45° from the horizontal axis of the bed. That is if the backrest does not reach a tilt of 45°, it is potentially uncomfortable to lie down and on the contrary, if it can be tilted below 45° it can be considered comfortable (therefore it is another rule of thumb).

Fig. 20
figure20

The poolside deckchair analogy

Let’s assume the analogy that the server is a poolside deckchair and subsequently the \(CiS^2\) index is analogous to the comfortability index. Thus, the analogous server \(CiS^2\) values should appear in the coordinate system formed by the backrest, the support of the backrest and the bed base (see Fig. 20). Then, the values for benchmarking two physical servers (or two different configurations for the same server) would be approximately represented in the backrest and the bed base (see green and orange variables in Fig. 20, zoom out is also provided). Therefore, let’s take an example of the two different poolside deckchair inclinations with different comfortability index, we may see the analogy for benchmarking servers through \(CiS^2\) values.

If both are two different scenarios for the same poolside deckchair, then the server scenario 1 (green representation) has a performance and energy trade-off consolidating homogeneous virtual machines more efficient than server scenario 2 (orange representation), because even though we may consider that the scalability vector of server would have n identical scalability vector sizes, the most efficient server is the green one with lower \(CiS^2_{h_1}\) of the \((h_1, CiS^2_{h_1})\) inflection point.

We know that this analogy about deckchair comfortability is a simplification of our proposal for an efficiency metric of consolidated virtual machines, since the shape of the representation of the \(CiS^2\) values is not exactly equal. However, we feel that poolside deckchair analogy inclination serves as a visual approximation of understanding the advantages of using \(CiS^2\) for benchmarking representation with different kind of servers.

Conclusions, implications and future work

In this paper, we propose a new index to quantify the performance-energy trade-off in VMC, the Consolidation index for CPU-Server Saturation (\(CiS^2\)). The proposed metric assists system administrators to take suitable decisions about the VMC. Also, the \(CiS^2\) reflects the changes in consolidated servers, capturing the raw data that feeds current datacentres.

We revisited different concepts about performance, energy consumption, and efficiency in virtualized systems, and how these are related. Then, we define the \(CiS^2\) index, as a metric to measure the trade-off between the performance and energy when consolidating virtual machines. The index was evaluated in a theoretical and in a practical manner. To evaluate the index theoretically, we take into account the features that an index should have to be representative of this particular case. From a practical point of view, we develop a set of experiments with different types of servers and CPU workloads, showing the adopted methodology and techniques.

From the experimental results, we can extract that the \(CiS^2\) index allows systems’ administrators to quantify how the performance-energy trade-off under a number of consolidated machines is. Our index representation may distinguish the more efficient configurations from other ones in a simple graphical manner, which is governed by two rules of thumb. However, the \(CiS^2\) index is computed in function of the number of consolidated machines, depicting the scalability of the system, in terms of consolidated virtual machines.

In addition, the \(CiS^2\) allows the system administrator studying and select the most efficient configurations through the proposed selection-algorithm. Also, it allows forecasting the behaviour of a system in future configurations. Moreover, in order to clarify the simplicity of our proposal, we use a poolside deckchair comfortability analogy to explain the \(CiS^2\) index usefulness.

As a future work, we can define some interesting lines of research. First, different properties of a system will be studied in function of the \(CiS^2\) index, as well as the elasticity. In addition, from the viewpoint of system administrators, we would implement an automatic tool in order to obtain the graphical representation of the \(CiS^2\). Also, the comparison between Type-1 hypervisor and the Type-2 one would be interesting for the \(CiS^2\) behaviour evaluation. To finish, we can apply the evaluation methodology to several devices, such as main memory and storage, to compute and study the effects of VMC.

References

  1. 1.

    Abaunza, F., Hameri, A., Niemi, T.: Eeui: a new measure to monitor and manage energy efficiency in data centers. Int. J. Prod. Perform. Manage. 67(1), 111–127 (2018)

    Article  Google Scholar 

  2. 2.

    Atrey, A., Jain, N., Iyengar, N.: A study on green cloud computing. Int. J. Grid Distrib. Comput. 6, 93–102 (2013)

    Article  Google Scholar 

  3. 3.

    Barroso, L.A., Hölzle, U.: The case for energy-proportional computing. Computer 12, 33–37 (2007)

    Article  Google Scholar 

  4. 4.

    Barroso, L.A., Clidaras, J., Hölzle, U.: The datacenter as a computer: an introduction to the design of warehouse-scale machines. Synth. Lect. Comput. Arch. 8(3), 1–154 (2013)

    Google Scholar 

  5. 5.

    Bermejo, B., Filiposka, S., Juiz, C., Gómez, B., Guerrero, C.: Improving the energy efficiency in cloud computing data centres through resource allocation techniques. In: Proceedings of the Research Advances in Cloud Computing, pp. 211–236. Springer (2017)

  6. 6.

    Bermejo, B., Juiz, C., Guerrero, C.: On the linearity of performance and energy at VMC: the CiS2 index for CPU workload in server saturation. In: Proceedings of the IEEE High Performance Computing and Communications (HPCC-2018) (2018)

  7. 7.

    Bermejo, B., Juiz, C., Guerrero, C.: Virtualization and consolidation: a systematic review of the past 10 years of research on energy and performance. J. Supercomput. (2018). https://doi.org/10.1007/s11227-018-2613-1

    Article  Google Scholar 

  8. 8.

    Bermejo, B., Juiz, C., Thomas, N.: On the virtualization overhead and energy consumption in consolidated servers. In: Proceedings of the UK-Performance Engineering Workshop (UKPEW) (2018)

  9. 9.

    Calheiros, R.N., Ranjan, R., Buyya, R.: Virtual machine provisioning based on analytical performance and qos in cloud computing environments. In: Proceedings of the 2011 International Conference on Parallel Processing (ICPP), pp. 295–304. IEEE (2011)

  10. 10.

    Cartwright, N., Bradhurn, N.: The possibility of a universal social welfare function. Report, London School of Economics

  11. 11.

    Casalicchio, E.: A study on performance measures for auto-scaling CPU-intensive containerized applications. Clust. Comput. 22, 995–1006 (2019)

    Article  Google Scholar 

  12. 12.

    Dayarathna, M., Wen, Y., Fan, R.: Data center energy consumption modeling: a survey. IEEE Commun. Surv. Tutor. 18(1), 732–794 (2016)

    Article  Google Scholar 

  13. 13.

    De Napoli, C., Forestiero, A., Lagana, D., Lupi, G., Mastroianni, C., Spataro, L.: Efficiency and green metrics for distributed data centers. Report P-26, ICAR (2016)

  14. 14.

    Ferreira, A.M., Pernici, B.: Managing the complex data center environment: an integrated energy-aware framework. Computing 98(7), 709–749 (2016)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Gonzalez, R., Horowitz, M.: Energy dissipation in general purpose microprocessors. IEEE J. Solid-State Circuits 31(9), 1277–1284 (1996)

    Article  Google Scholar 

  16. 16.

    Hsu, C.-H., Poole, S.W.: Revisiting server energy proportionality. In: Proceedings of the 2013 42nd International Conference on Parallel Processing (ICPP), pp. 834–840. IEEE (2013)

  17. 17.

    Huber, N., von Quast, M., Hauck, M., Kounev, S.: Evaluating and modeling virtualization performance overhead for cloud environments. In: Proceedings of the CLOSER, pp. 563–573 (2011)

  18. 18.

    Hwang, K., Bai, X., Shi, Y., Li, M., Chen, Wen-Guang, Yongwei, Wu: Cloud performance modeling with benchmark evaluation of elastic scaling strategies. IEEE Trans. Parallel Distrib. Syst. 27(1), 130–143 (2016)

    Article  Google Scholar 

  19. 19.

    Jain, A., Mishra, M., Peddoju, S.K., Jain, N: Energy efficient computing-green cloud computing. In: Proceedings of the 2013 International Conference on Energy Efficient Technologies for Sustainability (ICEETS), pp. 978–982. IEEE (2013)

  20. 20.

    Jain, R.: The Art of Computer Systems Panalysis: Techniques for Experimental Dmeasurement, Simulation, and Modeling. Wiley, Boca Raton (1990)

    Google Scholar 

  21. 21.

    Jiang, C., Wang, Y., Ou, D., Li, Y., Zhang, J., Wan, J., Shi, W.: Energy efficiency comparison of hypervisors. In: Proceedings of the Sustainable Computing: Informatics and Systems (2017)

  22. 22.

    Kaur, T., Chana, I.: Energy efficiency techniques in cloud computing: a survey and taxonomy. ACM Comput. Surv. (CSUR) 48(2), 22 (2015)

    Article  Google Scholar 

  23. 23.

    Kueng, P.: Process performance measurement system: a tool to support process-based organizations. Total Qual. Manage. 11(1), 67–85 (2000)

    Article  Google Scholar 

  24. 24.

    Lovász, G., Niedermeier, F., De Meer, H.: Performance tradeoffs of energy-aware virtual machine consolidation. Clust. Comput. 16(3), 481–496 (2013)

    Article  Google Scholar 

  25. 25.

    Minas, L., Ellison, B.: Energy Efficiency for Information Technology: How to Reduce Power Consumption in Servers and Data Centers. Intel Press, Hillsboro (2009)

    Google Scholar 

  26. 26.

    Molero, X., Juiz, C., Rodeño, M.J.: Evaluación y modelado del rendimiento de los sistemas informáticos. Prentice Hall, Upper Saddle River (2004)

  27. 27.

    Monteiro, A., Loques, O.: Quantum virtual machine: power and performance management in virtualized web servers clusters. Clust. Comput. 22(1), 205–221 (2019)

    Article  Google Scholar 

  28. 28.

    Munteanu, I., Debusschere, V., Bergeon, S., Bacha, S.: Efficiency metrics for qualification of datacenters in terms of useful workload. In: Proceedings of the PowerTech (POWERTECH), 2013 IEEE Grenoble, pp. 1–6. IEEE (2013)

  29. 29.

    Muraña, J., Nesmachnow, S., Armenta, F., Tchernykh, A.: Characterization, modeling and scheduling of power consumption of scientific computing applications in multicores. Clust. Comput. (2019). https://doi.org/10.1007/s10586-018-2882-8

    Article  Google Scholar 

  30. 30.

    Panda, S.K., Jana, P.K.: An energy-efficient task scheduling algorithm for heterogeneous cloud computing systems. Clust. Comput. 22, 509–527 (2018)

    Article  Google Scholar 

  31. 31.

    Prakash, S.J., Subramanyam, K., Prasad, U.D.S.V.: Towards energy efficiency of green computing based on virtualization. Int. J. Emerg. Trends Eng. Dev. 7(2), 415–423 (2012)

    Google Scholar 

  32. 32.

    Reddy, V.D., Setz, B., Rao, G.S.V., Gangadharan, G.R., Aiello, M.: Metrics for sustainable data centers. IEEE Trans. Sustain. Comput. 2(3), 290–303 (2017)

    Article  Google Scholar 

  33. 33.

    Reddy, V.D., Setz, B., Rao, G.S.V., Gangadharan, G.R., Aiello, M.: Live migration in bare-metal clouds. IEEE Trans. Cloud Comput. 1(1), 99 (2018)

    Google Scholar 

  34. 34.

    Sen, R., Wood, D.A.: Energy-proportional computing: a new definition. Computer 8, 26–33 (2017a)

    Article  Google Scholar 

  35. 35.

    Sen, R., Wood, D.A.: Pareto governors for energy-optimal computing. ACM Trans. Arch. Code Optim. (TACO) 14(1), 6 (2017b)

    Google Scholar 

  36. 36.

    Tang, C.-J., Dai, M.-R., He, H.-C., Chuang, C.-C.: Evaluating energy efficiency of data centers with generating cost and service demand. Bull. Netw. Comput. Syst. Softw. 1(1), 16 (2012)

    Google Scholar 

  37. 37.

    Uddin, M., Rahman, A.A.: Server consolidation: an approach to make data centers energy efficient and green. arXiv Preprint. arXiv:1010.5037 (2010)

  38. 38.

    Uddin, M., Rahman, A.A.: Energy efficiency and low carbon enabler green it framework for data centers considering green metrics. Renew. Sustain. Energy Rev. 16(6), 4078–4094 (2012)

    Article  Google Scholar 

  39. 39.

    Vasan, A., Sivasubramaniam, A., Shimpi, V., Sivabalan, T., Subbiah, R.: Worth their watts? An empirical study of datacenter servers. In: Proceedings of the 2010 IEEE 16th International Symposium on High Performance Computer Architecture (HPCA), pp. 1–10. IEEE (2010)

  40. 40.

    Ventre, P.L., Lungaroni, P., Siracusano, G., Pisa, C., Schmidt, F., Lombardo, F., Salsano, S.: On the fly orchestration of unikernels: tuning and performance evaluation of virtual infrastructure managers. IEEE Trans. Cloud Comput. (2018)

  41. 41.

    Volk, E., Tenschert, A., Gienger, M., Oleksiak, A., Sisó, L., Salom, J.: Improving energy efficiency in data centers and federated cloud environments: comparison of coolemall and eco2clouds approaches and metrics. In: Proceedings of the 2013 Third International Conference on Cloud and Green Computing (CGC), pp. 443–450. IEEE (2013)

  42. 42.

    von Kistowski, J., Block, H., Beckett, J., Spradling, C., Lange, K.-D., Kounev, S.: Variations in CPU power consumption. In: Proceedings of the 7th ACM/SPEC on International Conference on Performance Engineering, pp. 147–158. ACM (2016)

  43. 43.

    Wang, B., Song, Y., Sun, Y., Liu, J.: Analysis model for server consolidation of virtualized heterogeneous data centers providing internet services. Clust. Comput. 22, 911–928 (2018)

    Article  Google Scholar 

  44. 44.

    Wang, L., Khan, S.U.: Review of performance metrics for green data centers: a taxonomy study. J. Supercomput. 63(3), 639–656 (2013)

    Article  Google Scholar 

  45. 45.

    Whitney, J., Delforge, P.: Scaling up energy efficiency across the data center industry: evaluating key drivers and barriers. Issue Paper No. IP, pp. 14–08 (2014)

  46. 46.

    Xu, C., Ma, X., Shea, R., Wang, H., Liu, J.: Enhancing performance and energy efficiency for hybrid workloads in virtualized cloud environment. IEEE Trans. Cloud Comput. (2018). https://doi.org/10.1109/TCC.2018.2837040

    Article  Google Scholar 

Download references

Acknowledgements

We thank the Hasso-Plattner Institut who provided part of the infrastructure to perform the experimentation needed for this work through the WEEVIL2 project.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Belen Bermejo.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Juiz, C., Bermejo, B. The \(CiS^2\): a new metric for performance and energy trade-off in consolidated servers. Cluster Comput 23, 2769–2788 (2020). https://doi.org/10.1007/s10586-019-03043-8

Download citation

Keywords

  • Metrics
  • Consolidation
  • Performance
  • Energy
  • Servers
  • Benchmarking