Truthful online double auction based dynamic resource provisioning for multi-objective trade-offs in IaaS clouds

Abstract

Auction designs have recently been adopted for static and dynamic resource provisioning in IaaS clouds, such as Microsoft Azure and Amazon EC2. However, the existing mechanisms are mostly restricted to simple auctions, single-objective, offline setting, one-sided interactions either among cloud users or cloud service providers (CSPs), and possible misreports of cloud user’s private information. This paper proposes a more realistic scenario of online auctioning for IaaS clouds, with the unique characteristics of elasticity for time-varying arrival of cloud user requests under the time-based server maintenance in cloud data centers. We propose an online truthful double auction technique for balancing the multi-objective trade-offs between energy, revenue, and performance in IaaS clouds, consisting of a weighted bipartite matching based winning-bid determination algorithm for resource allocation and a Vickrey–Clarke–Groves (VCG) driven algorithm for payment calculation of winning bids. Through rigorous theoretical analysis and extensive trace-driven simulation studies exploiting Google cluster workload traces, we demonstrate that our mechanism significantly improves the performance while promising truthfulness, heterogeneity, economic efficiency, individual rationality, and has a polynomial-time computational complexity.

1 Introduction

The global cloud infrastructure services market is rapidly proliferating enterprise IT. According to a study by Mckinsey, public cloud expenditure is expected to grow six times the rate of general IT expenditure by 2020. The Infrastructure-as-a-Service (IaaS) cloud model is increasingly being adopted across several industry verticals to meet cloud user’s real-time demands through virtualization technologies to virtually access computing resources, i.e., storage, networking-related services, or processing power at cloud service providers (CSPs) such as Rackspace, Amazon EC2, VMware, Salesforce, Google cloud, and Microsoft Azure platform. Hypervisor-based virtualization technologies such as VMware ESXi, XenServer, KVM, and Microsoft’s Hyper-V have become very popular in recent years and massively applied in cloud infrastructures, explicitly, in IaaS. The key benefits of virtualization include availability, hardware independence, isolation, and security. CSPs offer cloud resources using virtual machines (VMs) to run applications (workload/services). Several other cloud services, such as Platform-as-a-Service (PaaS), Container-as-a-Service (CaaS) and Software-as-a-Service (SaaS), are most often deployed on top of an IaaS platform with all their applications running inside the VMs. However, in the data center, resources are often severely under-utilized; and substantial energy saving can be achieved through efficient resource management strategies, i.e., resource allocation, scheduling, and consolidation [1–4].

Fig. 1

Green cloud computing

This paper addresses the resource allocation problem in IaaS clouds. Efficient resource allocation problem is very challenging to IaaS cloud because, to satisfy the cloud users’ requests of desired resources (i.e, Memory, CPU, network bandwidth, and storage), it has to balance multi-objective trade-offs between CSP’s revenue and energy cost minimization while maintaining the performance as shown in Fig. 1. To maximize revenue, CSP prefers to allocate as many VMs as possible. However, it will drastically increase the power consumption due to a large number of active physical machines (PMs) [5].

In cloud trading markets, major IaaS providers today typically sell cloud resources via pre-configured VM instances of fixed types, while cloud customers acquire resources for executing tasks. Every participant in such a cloud computing market focuses on maximizing its revenues. Thus pricing is another critical factor for balancing the benefits of cloud users as well as CSPs. In trading, an efficient pricing strategy is required to maximize the individual profits of each participant. Cloud computing providers offer multiple pricing strategies, e.g., Microsoft Azure [6] provides resellers pricing, and Pay-As-You-Go (PAYG) models. Amazon EC2 [7] offers on-demand, reserve, spot pricing, and savings models. Most of the CSPs have adopted a fixed price strategy, i.e., charging customers a fixed amount for each pre-configured VM, where the total payment is calculated by the length of time, the total number of purchased VMs and the unit price. Despite the apparent simplicity in implementation, fixed pricing methods have failed to reflect the dynamic demand and supply relationship of the cloud market. As a result, it lacks efficiency and market agility compromising the CSP’s profit and customers’ utility. The adoption of different pricing models by multiple service providers poses the challenge to deliver cloud services at a reasonable price to users. Hence, in such a a federated cloud environment, we need a fair resource allocation technique to effectively allocate CSP’s resources to those cloud users who can value them the most and without any market manipulation.

In economics, the auction [8] is become one of the popular trading forms as it maintains efficiency and fairly distributes the resources to those service users who will value them most and without any market manipulation. Unfortunately, we have observed that only simple auction mechanisms have been implemented by leading CSPs. Such simple auctions have significant limitations as follows:

• 1. One-sided interactions: In auctioning, one-sided interaction means an auction can either be performed by a buyer-side (i.e., cloud users) or the seller-side (CSPs). For example many e-commerce platforms following one-to-many negotiations [9]. In a buyer-side auctioning, a buyer procures asks from multiple sellers and determines a winner to buy the commodity. While in a seller-side auctioning, a single seller offers its distributed commodities to multiple buyers.

• 2. Offline setting: In an offline setting, all users’ demands and bids are known in advance without considering the dynamics of an elastic model in the time-variant environment such as the arrival of users at some future time for any specified time interval, and time-based server maintenance in data centers. This scenario gives the advantage of acquiring higher social welfare, however the cloud service users have to wait for some period of time prior to the actual resources are assigned. Also, in the time-varying resource allocation users’ resource demands can not be consistent over time. For example, amid the COVID-19 lockdown, there is a sudden surge in traffic for video streaming platforms compared with the regular days. Another example of the Cab booking App server, which may not require many resources for data processing at night. This is because there are more users on the network during the peak period than the off-peak period.

• 3. Homogeneous task model: Existing double-auction techniques such as McAfee [10] was originally designed for homogeneous task model and not suitable for IaaS clouds having heterogeneous distribution of resources. Also, most of the studies have considered only a single type of VM, while the IaaS CSPs such as Microsoft Azure [6] and Amazon EC2 [7] offer heterogeneous types of VM instances.

• 4. One-to-one matching: Existing auction models such as in [11], restricted the final outcome of auction setting with one-to-one matching, where one buyer is matched to at most one seller and each seller will be alloted to at most one buyer.

• 5. Selfish strategies: Selfish strategies followed by cloud users or potential misreports of private information, such as the time-window of resource requirements and also the valuation, to obtain higher utilities. For example, in the mobile device cloud application model [12], some of the selfish mobile devices intended to maximize their utility by manipulating the claimed cost. Also, in the federated cloud environment, each selfish cloud provider desire to gain higher profit and total social welfare by declaring false information about the resource types and quantities [13].

To overcome the aforementioned drawbacks, we propose an Online Truthful Double Auction Mechanism called ’OTDAM’ for efficient resource allocation. OTDAM presents a more realistic case of the online double auction for IaaS clouds, with the unique characteristics of revenue maximization, energy cost minimization while performance is maintained. To overcome all the limitations of the current auction mechanisms, we consider to use a double auction or two-sided cloud market over a single-sided auction. The reason behind the selection is, a single-sided auction could be either a seller’s side that compete to sell their goods or the buyer’s side that compete to obtain a finite resource. But in reality, a CSP can sell IaaS resources to multiple service users, and users can purchase VMs of various types together from CSPs. As the CSP dynamically packs the requested VMs with numerous types of resources on heterogeneous PMs (cloud servers), it is more difficult to find an optimum provisioning decision under different server operational costs.

To solve this problem, we merge together a seller’s side as well as a buyer’s side auction design to devise a two-sided auction market framework, commonly referred to as ’double auction’ for many-to-many negotiations. A double auction strategy is an efficient trading technique for organizing trade in two-sided market actively involving service users to bid and CSPs to ask with their individual demands. For example, the leading foreign exchange (FX), New York Stock Exchange (NYSE), and NASDAQ employ double auctions variants [14]. Due to the participation of multiple sellers (servers) and buyers (cloud users), our proposed scenario ideally matches to double auction market. Thus, we model the problem of resource provisioning in cloud as a truthful online double auction mechanism, where our action obtains a maximal matching between the set of cloud servers and the service users in IaaS clouds. This work proposes a completely different approach of online auctioning for IaaS resource trading with the unique properties of elasticity for time-varying arrival of cloud users’ demands requesting for heterogeneous VMs and modeling the operational cost of servers under time-based maintenance in data centers. The auction mechanism design consists of a winning bid algorithm to find matching between cloud service users and servers and a payment algorithm applying price function to determine how much the cloud providers charge for compute instances of each cloud user.

1.1 Motivation

As demonstrated in Fig. 2, we focus on a two-sided IaaS cloud environment with cloud service users and service providers. Our study is driven by realistic cloud users demand and service provisioning scenarios where both cloud users and service providers are dynamic. It means the cloud service users may arrive and leave the system dynamically. Also, the physical machines (PMs) or servers in cloud data centers may have to undergo periodic maintenance, which may interrupt the services. Given the arrival-departure of cloud users and maintenance duration of the PMs, we need to perform matching of service demands, i.e., VMs to the PMs. After the arrival, service users submit their demands in the form of bids comprising start-time of VM, end-time of VM, resource requirement, and unit valuation for acquiring one-unit capacity in one-time slot.

As shown in the example, the service user is requesting a VM from 5:00 pm to 7:00 pm. The unit valuation is $2.00. Once the cloud service provider receives a bid, first, it determines the type of allocated VMs based on computation, memory, and storage capability. It then performs matching of user-requested VMs to the PMs while considering the time-varying arrival of demands and maintenance-duration of servers.

Fig. 2

Illustration of two-sided online interactions between cloud users and service providers

1.2 Contributions

The main technical contributions of the proposed work are as follows:

• We model the problem of dynamic resource allocation in IaaS clouds as an online double auction that offers an elastic framework for time-varying cloud user demands and availability of physical machines (PMs) due to periodic, or time-based maintenance in cloud data centers.

• We propose an online truthful double auction mechanism (OTDAM) consisting of a weighted bipartite matching based winning-bid algorithm for resource provisioning and a Vickrey–Clarke–Groves (VCG) driven algorithm for payment calculation of winning bids.

• Through rigorous theoretical analysis, we prove that OTDAM achieves truthfulness, economic-efficiency, individual rationality, and polynomial-time computational complexity.

• Based on the extensive trace-driven simulation study, we show that OTDAM maximizes revenue and minimizes energy cost while maintaining high performance in cloud data centers compared with the existing auction approaches.

1.3 Organization

The rest of the paper is structured as follows: In Sect. 2, we provide related works on double auction and resource allocation in IaaS clouds. In Sect. 3, we discuss the system model and problem formulation for dynamic resource provisioning problem. In Sect. 4, we present working of OTDAM, algorithms, an illustrative example, and also provide the proofs of desirable auction properties. We evaluate the approaches through extensive simulation studies in Sect. 5. Finally, Sect. 6 provides conclusions with some of the directions for future research.

2 Related works

This section presents some related works on double auction techniques and resource allocation approaches in IaaS clouds.

2.1 Double auction

In [15], Kumar et al. presented a comprehensive survey of double-auction approaches and investigate the benefits of applying double-auction in cloud markets for the trading of computing resources. Kumar et al. [16] further proposed a truthful combinatorial double-auction technique referred to as ’TCMDAC’ for pricing and resource allocation of computing resources in a cloud computing market. Farajian et al. [17] provided a continuous double-auction strategy for efficient resource allocation in the cloud market. In economics literature, McAfee [10], and Vickrey–Clarke–Groves (VCG) [18–20] are two well-known double-auction models. McAfee is based on the Trade-Reduction (TR) method, and it can achieve the properties of individual rationality, budget-balance, and truthfulness (incentive compatibility). While, the VCG based double-auction model can fulfill the property of individual rationality [11], and it can also satisfy the truthfulness property [21].

The McAfee [10] auction method can be illustrated as follows: Firstly, the auctioneer sorts the bids of buyers in non-increasing order, where $B_1 \ge B_2 \ge B_3...\ge B_m$. The ask of sellers’ are sorted in non-decreasing order: $A_1 \le A_2 \le A_3...\le A_n$. Here, the efficient number of trades is the number $k \le min\{n,m\}$, where $B_k \ge A_k$ and $B_{k+1} < A_{k+1}$. Finally, the median can be defined as $p_0 = \frac{A_{k+1}+B_{k+1}}{2}$. The McAfee needs a budget balancer to operate. If $A_k \le p_0 \le B_k$, then all efficient k buyers and sellers trade for the resource at price $p_0$. Otherwise, only $k-1$ buyers and sellers trade for $k-1$. In this case buyers pay $B_k$ and the sellers recieve ask $A_k$.

Generalized VCG double auction mechanism has also received popularity as they are efficient and strategy-proof. Without the budget-balance problem, the VCG payment scheme supports efficient, individual rationality, and strategy-proof exchange. VCG optimizes social welfare while achieving truthfulness property. It achieves budget-balance property by distributing “surplus” to each agent [21]. During VCG based auctioning, if $B_i \le A_j$, then no trade is performed. If $B_i > A_j$, then only the trading is performed, and the buyer pays the ask $A_j$, and the seller receives the bid $B_i$. The McAfee and VCG auction mechanisms can not be directly applied for the proposed scenario due to the following limitations:

• (i) McAfee provides a simple exchange environment for single-unit resources only, where sellers and buyers trade for single-units of the same good, and consequently it is not applicable for IaaS clouds having multi-unit resources.

• (ii) McAfee’s mechanism is weakly budget-balanced.

• (iii) The McAfee [10] double-auction technique was originally designed for homogeneous commodities where a buyer can be assigned to one seller only. But it is not suitable for IaaS clouds having heterogeneous commodities and need multiple assignments of a seller.

• (iv) VCG payment scheme is the most efficient truthful mechanism. However, it does not ensure the criteria of budget-balancing. Subsequently, the auctioneer actually has to subsidize the trade.

Table 1 Comparison of existing double auction approaches in the economics literature

Huang et al. [22] designed a generalized model of McAfee [10] for a multi-unit trading environment, which is proven to be budget-balanced, truthful, and individually rational. Babaioff et al. [23] proposed an extended Trade Reduction (TR) technique for single unit commodities. It performs the bidding of buyers in bundles, and each seller sells one unit of each commodity. Chu and Shen [24] developed a multi-stage and truthful double auction approach for the consumer to consumer (C2C) market environment. The authors have assumed the single-unit market where a consumer bids for a bundle of commodities, and the seller sells a single unit for a single commodity. The mechanisms are proven to be truthful, budget-balanced, individually rational, and asymptotic efficient. Chu and Shen [25] also designed 2-incentive compatible double auction-based multi-stage mechanisms named as buyer-competition, and modified-buyer-competition). The authors have assumed a single-unit trading market where a buyer acquires the commodities in bundles with a seller selling a unit capacity for a single type of commodity. The approaches are further proven to be budget-balance and incentive compatibility.

Chu [26] provided a padding approach in a multi-unit environment, which is highly efficient than the single-unit schemes. The double auction mechanisms are proven to be budget balanced, incentive compatible, and asymptotic efficient. Mishra et al. [27] designed a descending price Dutch auction mechanism for multiple heterogeneous items. Demange et al. [28] proposed two dynamic auction mechanisms to achieve equilibrium and degree of accuracy. Ausubel et al. [29] presented a dynamic auction approach for multiple heterogeneous commodities. Charles et al. [30] devised a multiple unit double-auction that enables numerous units or block trades in the double-auction framework. Yang et al. [11] designed TASC auctioning for cooperative communication, where the wireless node can participate in relay services trading. The overall comparison of the aforementioned double-auction mechanisms is shown in Table 1.

2.2 Resource allocation in IaaS clouds

Table 2 Comparison of existing auction techniques in cloud literature

Jin et al. [38] provided cloudlet based resource sharing auction schemes for mobile cloud computing. However, these auction techniques are only applicable for homogeneous task models, and resultant matching for auctioning is one-to-one. Wang et al. also [12] proposed two truthful auctioning schemes for task based allocation in mobile clouds. They presented a VCG-driven truthful auction approach in the homogeneous cloud environment, and also a winning-bid algorithm for heterogeneous cloud models. In [31–35], and [37] CSPs adopted the auction-based resource allocation model as an economic paradigm to provide VM resources to the worthy users. In this design, the cloud users submit their resource demands with their values to the CSP. Then the CSP determines the allocation of VMs to the users for maximizing revenue. In [32], Nejad et al. given a VCG-based truthful auction scheme for revenue maximization, and prove that it is an NP-hard optimization problem.

However, to deal with the dynamic environment, Mashayekhy et al. [31] and Zhang et al. [34] developed an online truthful scheme that is invoked during the arrival of user requests. Zhang et al. [35] further enhance the online mechanism by giving more flexibility for user bidding. It means if the required resources are accessing during a particular time slot, then a user request can be accommodated. Wang et al. [36] consider an efficient VM allocation problem for reducing energy consumption. Zaman et al. [37] presented the VM allocation problem as a combinatorial auction problem. They have provided combinatorial auction-based approaches and compare with fixed-price allocation approach. Jin et al. [38] designed a truthful incentive mechanism (TIM) for auctioning of resources between the cloudlets and mobile devices. Lu et al. [39] suggested a truthful double auction scheme for bridging the gap between cloud users’ task demands and service providers’ resources for two-sided markets.

Zhang et al. [40] provided an online auction method for time-varying assignment and pricing of multidimensional resources in the cloud environment. Middya et al. [41] developed ’TARA’, a multi-unit double auction approach for resource provisioning in a federated cloud environment. The authors have considered a multi-buyer and multi-seller double auction design for heterogeneous cloud resources. Patel et al. [42] designed ’TDAM’, a truthful double auction scheme to provision VMs in cloud data centers. Jin et al. [43] developed ’ICAM’ an incentive-compatible auction approach to assign cloudlets in order to fulfill the service requirements of mobile devices.

However, the main problem with the reported works is the lack of efficient pricing strategies for minimizing the cost of energy consumption and maximum revenue realized by satisfying the arriving user’s demands. In comparison to the existing techniques, the auctioning problem studied in this work mainly differs in the following aspects: (i) heterogeneity of cloud environment where VMs have different resource requirements and PMs with different resource availability; (ii) double auction mechanism to consider the dual preferences where VMs act as buyers and PMs act as sellers; (iii) truthful auctioning while considering the fundamental research issues of revenue maximization and energy cost minimization, both are critical objectives for VM allocation scheme; (iv) dynamics of an elastic model in the time-variant environment such as arrival and departure of users and time-based server maintenance in data centers. Table 2 presents a comparison between the proposed approach and the closely related works in 12 different aspects.

3 System model and problem formulation

3.1 Auction model

We model the dynamic resource provisioning problem as an online double auction model, as illustrated in Fig. 3.

Fig. 3

Auction model

The auction model comprises of two-sided interactions between cloud service users and CSP. The auctioneer handles the auction process and acts as a trusted third party between cloud service users called buyers, and servers of CSP called sellers. Note that we will use buyers interchangeably with cloud service users and sellers with servers of a cloud service provider. We assume that the cloud service users request for VM instances. CSP offers a wide variety of instance types including varying combinations of memory, CPU, networking capacity, and storage. The CSP allocates the requested resources to the servers or physical machines (PMs). Cloud service users may arrive and leave the system dynamically. They submit their demand to reserve the resources for future use. A cloud service user submits a resource request in the form of a bid consisting of the service-time window i.e, start and finish times to use VMs, resource requirement, and the valuation or minimum price per unit time for getting one unit capacity. All the participants submit bids and asks to the cloud auctioneer in such a manner so that no auction participant has any idea about a realistic bids/asks of any other participants. With the help of received bids and asks values, an auctioneer discovers mapping between the PMs of CSP and the cloud service users. The auctioneer also determines the hammer price (clearing-price) and the final payment. In the cloud market, a CSP can deliver the cloud services to users if and only if the auctioneer discovers a suitable matching between that particular CSP and the service user. Now, let us understand the entities involved in the auction process.

3.1.1 Sellers (i.e., Physical machines (PMs))

Cloud service providers offer $\kappa$ servers or PMs to fulfill the demands of $\eta$ VM instances in a cloud data center. A VM instance must be placed to at most one cloud server. Also, it cannot operate on multiple resources provided by multiple servers. Each seller can be expressed as a 11-tuple $S_j= <\alpha _j,\sum _j,\psi _j,\phi _j,\tau _j, a_j,d_j,\chi _j,$ $\epsilon _j^{id},\epsilon _j^a,v_j>$, where $\alpha _j$ represents the ask-per-resource submitted by $S_j$, $\sum _j$ indicates the list of hosted VMs at the particular cloud server, $\psi _j$ implies the total resources available at the server, $\phi _j$ denotes the available resources, $\tau _j$ refers the response time of server, $\chi _j$ represents the payment the seller will gain when the auction is completed, $a_j \in T$ and $d_j \in T$ denote the arrival and departure time respectively and $v_i$ represents the valuation of the ask raised by the Seller. $\epsilon ^a_j$ indicates the energy consumption of the fully utilized server and, $\epsilon ^{id}_j$ denotes the energy consumption of the fully idle server.

3.1.2 Buyers (i.e., The cloud users)

We assume a set of $\eta$ buyers $\ B=\{B_{1}, B_{2}, \dots , B_{\eta }\}$ requesting resources from $\kappa$ servers $\ S=\{S_{1}, S_{2}, \dots , S_{\kappa }\}$. A user request $C \in B$ arrives at slot t and sends a bid to the auctioneer. A user specifies the VM requirement according to its own needs and hence will report honestly. A server after joining the system dynamically will raise an ask per resource demanded by the buyer. Each buyer can be expressed as 7-tuple, $B_{i}= <\beta _i,\mu _i,\varUpsilon _i^m,\chi _i,a_i,d_i,v_i>$ where $\beta _i$ denotes the bid per resource which $B_i$ places, $\mu _i$ represents the server allocated to $B_i$, $\varUpsilon _i^m$ is the type m VM resource requirement and $\chi _i$ denotes the total payment the buyer pays after auction is over, $a_i \in T$ and $d_i \in T$ indicate the start-time and finish-time of the buyer respectively, $v_i$ denotes the valuation of the bid raised by buyer.

3.1.3 Auctioneer

The cloud auctioneer acts as a trustworthy third-party, which decides the mapping between the dynamic set of winning buyers ${\mathbb {B}}(t) \in B(t)$ and the set of winning sellers ${\mathbb {S}}(t) \in S(t)$ at time t, i.e, $\sigma : \{i: B_i \in {\mathbb {B}}\} \rightarrow \{j: S_j \in {\mathbb {S}}\}$. The auctioneer carries the task in slots and hence the set $T= \{$0, 1,...., t$\}$. The auction process is performed into three phases, as follows:

• Platform setup Cloud users (buyers) and servers (sellers) details are taken as input and arranged accordingly.

• Winner-determination and matching To assign the winning cloud servers according to their asking and energy, response-time values to the winning cloud users as per their bids.

• Payment schedule Using a VCG-based payment strategy, we determine the payment $\phi _i^B$ the winning buyer ${\mathbb {B}}(t) \in B$ is charged and also the payment $\phi _j^S$ the winning seller ${\mathbb {S}}(t) \in S$ is rewarded per-unit of resource.

3.2 Assumptions

We make some practical assumptions for online auction model as follows:

• 1. Buyers & sellers have bounded patience measured as K slots of waiting time, i.e. $a_i + K \ge d_i$.

• 2. The auctioneer holds the resource allocation and forwards it to the buyer when the buyer departs. It also contains the payment to the server and pays it when it leaves.

• 3. The cloud users and servers have the same bids and asks respectively for the same type in the heterogeneous environment.

In Table 3, we have summarized some important mathematical notations used in this paper.

Table 3 Main notations and their descriptions

3.3 Utility functions

Definition 31

(Buyer Utility Function) For a buyer (cloud service user) $B_i \in {\mathbb {B}}$ the utility represents the difference between the true valuation of the bid raised and price paid to the auctioneer.

$$\begin{aligned} {\mathbb {U}}_{i}^B = \left\{ \begin{array}{lr} \varUpsilon _{i}{v_{i}} - \chi _{i} , &{} \text {if } B_{i} \in {\mathbb {B}}\\ 0 , &{} \text {otherwise}\\ \end{array}\right\} \end{aligned}$$

(1)

where $v_{i}$ is the valuation of the buyer’s instance. $v_{i}$ states the buyer’s true cost who makes willingness for payment of a single resource usage.

Definition 32

(Seller Utility Function) For a seller (PM) $S_i \in {\mathbb {S}}$ the utility can be characterized as the difference between payment received from the auctioneer and actual cost to perform resource allocation.

$$\begin{aligned} {\mathbb {U}}_{j}^S = \left\{ \begin{array}{lr} \chi _{j}-(\phi _{j}-\psi _{j})v_{j}, &{} \text {if } S_{j} \in {\mathbb {S}}\\ 0 , &{} \text {otherwise}\\ \end{array}\right\} \end{aligned}$$

(2)

where $v_{j}$ is the valuation of the seller’s instance. $v_{j}$ tells the seller’s true cost who has to sustain to provide a single resource to the buyer.

Definition 33

(Auctioneer Utility Function) The utility of a matched pair of a cloud service user to a server in a particular time slot is the payment difference between the buyer to the seller.

$$\begin{aligned} {\mathbb {U}}_{i}^A = \left\{ \begin{array}{lr} \chi _{i}-\chi _{j}, &{} \text {if } \beta _{i} \ne \alpha _{i}\\ 0 , &{} \beta _{i} = \alpha _{i}\\ \end{array}\right\} \end{aligned}$$

(3)

where ${\mathbb {U}}_{i}^A$ is utility of auctioneer.

3.4 Problem formulation

Our objective is to maximize the revenue through the allocation of VM instances and minimize the energy consumption of cloud servers by examining the energy performance trade-off under the allocation and resource constraints, while considering the time-varying arrival of the cloud service users during the server maintenance period. Also, we aim to achieve the desired double auction features, such as truthfulness, economic efficiency, and individual rationality. Next, we mathematically describe the optimization problem and then introduce the properties of double auction.

Definition 34

(Revenue Maximization) Let $\beta _t$ is the set of bids received by time ${\widetilde{t}}$, $\varGamma _t= \{\beta _t|t_i\le {\widetilde{t}}\}$. Then, the total number of sold type-m VMs till time slot ${\widetilde{t}}$ is:

$$\begin{aligned} \sum _{i=1}^{\eta -1}\varUpsilon _i^mp_{i}(t) + \varUpsilon _{\eta }^m\beta _{\eta }(t), {\widetilde{t}}\le t \le T. \end{aligned}$$

(4)

Next, we formulate the revenue maximization objective as follows:

$$\begin{aligned} maximize \sum _{t=1}^{T}\Bigg [\sum _{i=1}^{\eta -1}\varUpsilon _i^mp_{i}(t) + \varUpsilon _{\eta }^m\beta _{\eta }(t)\Bigg ], \forall {m \in M}, \forall {t} \in [1,T] \end{aligned}$$

(5)

where,

$$\begin{aligned} p_{i} = \left\{ \begin{array}{lr} \alpha _{\mu _{i}}, &{} \text {if } \alpha _{\mu _{i}}\ge \beta _{\theta +1}\\ \beta _{\theta +1}, &{} \text {otherwise}\\ \end{array}\right\} \end{aligned}$$

(6)

Here $\theta$ represents the position of buyer $B_{i}$ in $\varTheta$, under the following constraints -

• 1. Resource capacity constraint The resource capacity constraint indicates that the allocated capacity of the total number of hosted type m VMs at a cloud server j must be less than or equal to the total number of available resources at server j of the data center. Implicitly, it is guaranteed that the allocation of resources for any type m VM will not reach beyond the capacity of server j.

  $$\begin{aligned} \sum _{k\in \sum _{j}}\varUpsilon _{k}^m\le \psi _{j}, \forall 1\le j\le \kappa , t \in T \end{aligned}$$

  (7)

• 2. Placement constraint The placement constraint ensures that the requested type m VM can be provisioned on at most one server at the cloud data center.

  $$\begin{aligned} \sum _{t \in T}\sum _{\mu \in S_{\kappa }}y_{\eta t \mu } \le 1, \forall B_{\eta }(t) \in {\mathbb {B}}, \mu \in [1, \kappa ], t \in T \end{aligned}$$

  (8)

Revenue can be maximized by increasing the cardinality of winning buyers set denoted by (${\mathbb {B}}$).

$$\begin{aligned} maximize \left| {{\mathbb {B}}}\right| : {\mathbb {B}}\subseteq B \end{aligned}$$

(9)

The fundamental idea behind the maximization of VM allocation is to always place the seller’s costly resources to the buyers having a higher valuation, who can afford to pay. Consequently, it will leave the economic seller resources for the buyers having a low valuation.

Definition 35

(Energy Minimization) The CSP’s operational cost mainly depends on the energy consumption cost. In fact, the power consumption of a data center increases linearly as the resource utilization increases. The total energy cost at time slot t can be stated as:

$$\begin{aligned} minimize \sum _{j=1}^{\kappa } (\epsilon ^{a}_{j}\varPi _{j}(t) + \epsilon ^{id}_{j}(1-\varPi _{j}(t))) \end{aligned}$$

(10)

where $\varPi$ represents the server utilization denoted as -

$$\begin{aligned} \varPi _{j}(t)=\frac{\psi _{j}(t)-\phi _{j}(t)}{\psi _{j}(t)}, t \in T \end{aligned}$$

(11)

Definition 36

(Performance Maximization) As suggested by Gupta et al. [44] “the ERP (Energy and Response-timeratio. Here the performance can Product) can be applied to acquire the trade-off between the performance and energy parameter”. By minimizing ERP, we can maximize the “performance-per-watt (PPW)” ratio. Here the performance can be denoted as the inverse of run-time or response–time denoted by R. The ERP value is calculated by Eq. 12:

$$\begin{aligned} ERP = \delta w_1 E \times \delta w_2 R \end{aligned}$$

(12)

where $\delta w_1$ and $\delta w_2$ represent the domination factors for E and R respectively [44, 45]. Hence the overall objective function for performance maximization can be written as:

$$\begin{aligned} minimize(ERP) \end{aligned}$$

(13)

3.5 NP-hardness

Theorem 1

The proposed online VM allocation problem is NP-Hard.

Proof

To prove that the online VM allocation problem is NP-Hard, let’s examine the problem of bin-packing. Assume the set of n items with different weights $w_{1}, w_{2},$ $w_{3}, \dots , w_{n}$ and bins each of with capacity c. Our goal is to place each item to a bin such that the number of total used bins is minimized. Next, we examine the proposed VM allocation problem. Assume a set of $\eta$ VM instances requiring heterogeneous resources $\varUpsilon _{1}^m, \varUpsilon _{2}^m, \varUpsilon _{3}^m, \dots , \varUpsilon _{\eta }^m$ and $\kappa$ servers with $\psi _{1}, \psi _{2}, \psi _{3}, \dots , \psi _{\kappa }$ capacities. The objective is to allocate each VM instance to a cloud server in such a way that the total number of allocated VM instances is maximized and the overall energy consumption of cloud servers is minimized. Now to prove a given problem is NP-Hard, it is sufficient to prove that its sub-problems are NP-Hard. So, our proposed problem can be further classified into two sub-problems as follows:

• 1. Maximize the number of assigned VMs: We can consider two cases of this problem as follows:

  • Case 1 When each cloud server is having unlimited amount of resources: VM instances can be allocated to the server by using the criteria of least ask-per-instance, let’s assume $S_{j}$, and now we require to pick the buyers satisfying the constraint $\beta _{i}\ge \alpha _{j}$. Then we compute the final buyers’ payment $\chi _{i}$ based on their corresponding bid-per-instance value, which can be performed with polynomial-time complexity.

  • Case 2 When each cloud server is having limited amount of resources: VM instances can be allocated to the server by applying the Algorithm 2 that is equivalent to assigning $\eta$ VM instances on $\kappa$ cloud servers in such a way that the total number of allocated VM instances are maximized. It also ensures the total number of active cloud servers are minimized, satisfying the allocation constraint and resource constraint. Thus, this problem is equivalent to the problem of Bin-Packing, which is NP-Hard. Afterward, we compute the final payment of buyers’ and sellers’ denoted as $\chi _{i}$ and $\chi _{j}$ respectively by applying the Algorithm 3, which can be solved with polynomial-time complexity.

• 2. Minimize the overall energy consumption of cloud servers: We can examine two cases of this problem as follows:

  • Case 1 When each cloud server is having unlimited amount of resources: VM instances can be allocated to the server having the least energy consumption, let’s assume $S_{j}$, and now we require to pick the buyers satisfying the constraint $\beta _{i}\ge \alpha _{j}$. Then we compute the buyers’ payment $\chi _{i}$ based on their corresponding bid-per-instance value, that would be performed with polynomial-time complexity.

  • Case 2 When each cloud server is having limited amount of resources: This case is just the same as the case 2 of the first sub-problem, except that we require to allocate the VMs to the cloud servers while considering the servers’ energy consumption which is equivalent to assigning $\eta$ VM instances to the $\kappa$ cloud servers in such a way that the overall value of $\sum _{j=1}^{\kappa }\oint$ is minimized satisfying the allocation constraint and resource constraint. Hence, this problem is equivalent to the decision version of the Bin-Packing problem, which is also a NP-Hard problem. Afterward, we can compute the final payment buyers’ and sellers’ denoted as $\chi _{i}$ and $\chi _{j}$ by applying the Algorithm 3, which can be executed with polynomial-time complexity.

  Since both these sub-problems are of the form of Bin-Packing problem, we can infer that proposed online VM allocation problem can be reduced in polynomial time to a Bin-Packing problem. Since it is known that the Bin-Packing is a NP-Hard problem, the online VM allocation problem is also NP-hard.

$\square$

3.6 Double auction properties

Designing an efficient double auction technique for allocating cloud resources of CSP to the cloud service users is expected to satisfy four desired properties as follows:

• 1. Truthfulness A truthful approach implies incentive compatibility, which means that the bid value of each buyer and the ask value of each seller must be equal to the actual true valuation and true cost of the resource, respectively. A property where the dominant strategy is bidding one’s true valuation, is called strategy proofs. A mechanism is truthful if a buyer or seller obtains the largest utility in a trade if and only if it truthfully represents its ask or bid information i.e.

  $$\begin{aligned} {\mathbb {U}}_{i}^B\ge {\mathbb {U}}_{i}^{{\hat{B}}}, {\mathbb {U}}_{i}^S \ge {\mathbb {U}}_{i}^{{\hat{S}}}, \forall {{\hat{B}}},{{\hat{S}}} \in \varOmega , \end{aligned}$$

  (14)

  where $\varOmega$ represents arbitrary information about an ask or a bid. Remark: We assume that the units (buyer/seller) cannot manipulate other task information, and truthful working only the bids and asks are considered. The unit (buyer/seller) cannot gain extra profit by reporting false information.

• 2. Individual Rationality: An auction design is individually rational if it does not incur a loss on joining the auction. This means a unit winning in the auction should have a non-negative utility and should gain profit, i.e.

  $$\begin{aligned} {\mathbb {U}}_{i}^B \ge 0, {\mathbb {U}}_{i}^S \ge 0, \chi _{i} \le \beta _{i}\nonumber \\ \chi _{j} \ge \alpha _{i}, \forall {B_{i} \in {\mathbb {B}} ,S_{j} \in {\mathbb {S}}}, \end{aligned}$$

  (15)

• 3. Budget-Balance An auction mechanism follows the budget-balance property if, for a matched pair of a buyer and seller, the auctioneer charges buyer no less than what it pays to the seller. It deals with the difference in payment between buyer and seller i.e., the utility of the platform should be non-negative.

  $$\begin{aligned} {\mathbb {U}}_{i}^A \ge 0 \end{aligned}$$

  (16)

  It can be divided into two types:

  • (a) Strong Budget-Balance - The auctioneer should not lose or gain money. Transactions should strictly happen between buyer and seller i.e.

    $$\begin{aligned} \sum _{i=1}^{\eta } \chi _{i} = \sum _{j=1}^{\eta } \chi _{j} \end{aligned}$$

    (17)

  • (b) Weak Budget-Balance - The auctioneer must not pay for the transaction to be successful, in other words it can gain profit but not suffer any loss i.e.

    $$\begin{aligned} \sum _{i=1}^{\eta } \chi _{i} \ge \sum _{j=1}^{\eta } \chi _{j} \end{aligned}$$

    (18)

• 4. Economic efficiency Economic efficiency implies that the winner of the auction should be the one who deserves or values the auction most after the trades have been completed.

However, the Myerson–Satterthwaite (MS) impossibility theorem states that, it is always impossible to accomplish all four of these desirable auction properties simultaneously.

4 Proposed approach

The detailed algorithm for proposed auction mechanism (OTDAM) is given by Algorithm 1. OTDAM works in two modules: winner-determination and matching and VCG based payment calculation. OTDAM calls Winner_Determination$(S, B, \eta , \kappa , T)$ to obtain the set of winner candidates (as shown in Algorithm 2). After winner determination stage, OTDAM performs final payment calculation using VCG_Payment_Calculation ($A, {\mathbb {B}}, {\mathbb {S}}, \varTheta , \varOmega$) (as shown in Algorithm 3). We next describe the winner-determination and the VCG based payment calculation algorithms in detail.

4.1 Winner determination and matching

To perform online double auction, first we form list B and S of the bids and asks of the buyers and sellers respectively, we get $\eta$ and $\kappa$ as :

$$\begin{aligned} \eta =|B| , \kappa =|S| \end{aligned}$$

(19)

Then we set the initial values of the parameters for buyer and seller. With each time slot, we check the condition of the buyer and seller availability for a particular time slot. We sort all the bids per resource of buyers in non-increasing sequence, so that buyer having the highest bid goes first to satisfy the resource request. A function $\sigma$ is a ’seller desirability’ or usability factor which denotes the ratio of the ask-per-resource($\alpha _{j}$) to the product of fully-utilized-energy-consumption ($\epsilon ^{a}_j$), when server is fully utilized, and response-time of the seller in milliseconds.

$$\begin{aligned} \sigma = \frac{\alpha _j}{\epsilon ^{a}_j*\tau _j} \end{aligned}$$

(20)

where $\tau _j$ is the response time. Therefore we have to sort all the sellers according to their $\sigma$ values in non-increasing order. So that we can allocate maximum number of possible VM instances to the cloud server having least active energy consumption value, and the cloud server with highest ask-per-resource considering the VM instances with the highest bid value are granted first. Thus we iterate over both the lists within a particular time slot to assign maximum possible VM instances to the cloud server.

We check for the availability of resources at the server and then allocate keeping in priority that the highest ask-per resource of the seller must be less than or equal to the buyer’s having lowest bid value. With each matching pair, we add the corresponding winner buyer and seller to their winning set ${\mathbb {B}}$ and ${\mathbb {S}}$, respectively.

4.2 VCG based pricing scheme

Vickrey–Clarke–Groves (VCG) pricing scheme is a generic truthful approach for achieving a socially-optimal solution. It is one of the most commonly used scheme for payment determination. Our payment schedule does not follow VCG completely, as VCG lacks in providing budget-balance over individual rationality. Next, we iterate the list of cloud users and find the final payment value for each user in the same time slot before it increments. Payment value is a product of resource requirement of the minimum value required to claim that cloud server, whose value is equal to the greater of the two i.e., ask-per-resource and the second-highest bid.

Considering $\vartheta$ as the current buyer’s position in the $\varTheta$ ordering list and the particular buyer placed at cloud server j.

$$\begin{aligned} \chi _{i} = \left\{ \begin{array}{lr} \varUpsilon _{i}\alpha _{j}, &{} \text {if } \alpha _{j}\ge \beta _{\vartheta +1}\\ \varUpsilon _{i}\beta _{\vartheta +1}, &{} \text {otherwise}\\ \end{array}\right\} \end{aligned}$$

(21)

We calculate payment the server will receive from buyer $B_{i}$, by computing minimum value from all the buyer’s payment ensuring that the designated buyer is hosted at next economical cloud server. Taking $\varepsilon$ as the current seller selected from the ordered list $A^{\alpha }$, we obtain

$$\begin{aligned} \chi _{j}^{i} = \left\{ \begin{array}{lr} \varUpsilon _{i}\beta _{j}, &{} \text {if } \beta _{i} \le \alpha _{\varepsilon +1}\\ \varUpsilon _{i}\alpha _{\varepsilon +1}, &{} \text {otherwise} \\ \end{array}\right\} \end{aligned}$$

(22)

The final payment to $S_{j}$ is the sum of all payments to $S_{j}$ received from all its buyers.

$$\begin{aligned} \chi _{j} = \sum _{i=1}^{|P|} \chi _{j}^{i} \end{aligned}$$

(23)

4.3 Illustrative example

We consider a random input of user requests and servers for a randomly generalized distribution of parameters assuming a valid time window and payment schedule. Table 4 shows the bid matrix that contains bids and resource requirements of 10 buyers. The ask vector for sellers is shown in Table 5. It contains ask, resource availability, active and idle energy consumption of 3 servers. The values of bids, asks, resource requirements, total resource availability, energy consumption, and response time have been generated between [2.0, 12.0], [1.0, 10.0], [3, 10], [10, 30], [150.0, 350.0], and [0.5, 2.0] respectively. Next, we illustrate the OTDAM steps to derive the auction outcome as follows:

Table 4 Bid matrix of 10 buyers

Table 5 Ask vector of 3 sellers

• Step 1 First, we form the ordered list $\varTheta$ of buyers, where all the buyers are sorted based on their per-resource bids in non-increasing order as displayed in Table 6.

• Step 2 Next, the sellers are sorted in the non-increasing sequence of their seller desirability values $\sigma$, as shown in Table 7.

• Step 3 Iterate the ordered list of buyers and sellers and allocate the VMs to the servers efficiently (Table 8).

• Step 4 Create the ordered list $A^{\alpha }$ for payment calculation (Table 9)

• Step 5 Calculate the payment for the buyers based on the resource demand and the minimum value required to claim that cloud server (Table 10).

• Step 6 Calculate the payment for the sellers (Table 11).

• Step 7 Now, we examine the individual rationality and truthfulness for the given example.

  • 1. Buyers’ Individual Rationality: As shown in Fig. 4, the actual payment a buyer has to give is never greater than the expected payment as per its bid.

  • 2. Sellers’ Individual Rationality: As represented in Fig. 5, the seller’s actual payment is never lesser than the expected payment as per its ask.

  • 3. Buyers’ Truthfulness: We maintain the auction’s truthfulness by ensuring that the buyer’s utility is maximum only when its bid equals the truthful cost. To analyze truthfulness, we select a random buyer, say buyer 8 and analyze its utility as it places bids different from its true value. Figure 6 shows that the utility increases when you get close to the truthful cost, is maximum at the truthful cost, and decreases from there. This is just an instance involving buyer 8 and the graph will not be the exact same for other buyers but will follow the same pattern due to the way the algorithm is designed (as shown in lines 23 and 24 of Algorithm 2).

• Step 8 The allocation done by OTDAM for a random input can be illustrated by a bipartite graph, as shown in Fig. 7. The vertices in the first row show VMs and the vertices in the second-row show servers. The randomly generated bid and resource requirement values are indicated above the buyer vertices, and the ask and total resource availability are indicated below the seller vertices. The numbers along the edge show the payment receiving from the buyers set and the payment obtained by the sellers set. Since we show this graph only for illustration of allocation, energy consumption values have not been stated. However, they are taken into account as displayed by the seller with ask value of 9 coming before the seller with ask value of 13.

Table 6 Build ordered list of buyers

Table 7 Ordered list of sellers based on their seller desirability values $\sigma$

Table 8 Allocation of VMs to the servers

Table 9 Build $A^{\alpha }$ for payment calculation

Table 10 Calculate buyers payment

Table 11 Calculate sellers payment

Fig. 4

Buyers’ individual rationality

Fig. 5

Sellers’ individual rationality

Fig. 6

Buyer’s truthfulness

Fig. 7

Bipartite graph representation of final VM allocation

4.4 Energy approximation

Definition 37

For any algorithm A(B, S), the asymptotic approximation ratio ($\nabla$) can be expressed as follows:

$$\begin{aligned} \nabla _{A}=\frac{A(B, S)}{OPT(B, S)} \end{aligned}$$

(24)

The total energy consumption can be calculated as:

$$\begin{aligned} \oint _{total}=\sum _{j=1}^{\kappa } (\epsilon ^{a}_{j}\varPi _{j}(t) + \epsilon ^{id}_{j}(1-\varPi _{j}(t))) \end{aligned}$$

(25)

For an ideal condition, an energy-optimal algorithm (EOA) does not consider the cloud server’s asking price, and try to assign VM instances in such a way that it consumes minimum energy. Hence, the overall energy consumption in this scenario can be written as:

$$\begin{aligned} \oint _{OPT}=\sum _{j=1}^{\kappa _{a}}\epsilon ^{a}_{j}+\sum _{j=\kappa _{a}+1}^{\kappa }\epsilon ^{id}_{j} \end{aligned}$$

(26)

where $\kappa _{a}$ represents the total active cloud servers. Here, we suppose that the cloud servers are positioned in an increasing sequence of their corresponding active energy consumption value and also in the decreasing sequence of their corresponding idle energy consumption value. Hence the $\oint _{OPT}$ can be expressed mathematically as follows:

$$\begin{aligned} \oint _{OPT}=\oint ^{a}+\oint ^{id} \end{aligned}$$

(27)

where $\oint ^{id}$ denotes the minimum idle energy consumption, and $\oint ^{a}$ represents the minimum active energy consumption.

Note that $\oint ^{a}$ and $\oint ^{id}$ are ideal values which more often are not achieved in the real world scenario. So, our algorithm will yield an energy consumption $\oint _{OTDAM}$ which will be calculated by taking the server utilization values $\psi$, into consideration.

Theorem 2

The maximum energy consumed by OTDAM, when $\varPi _{j}(t)=1$, $\forall {1\le j\le \kappa }$ is calculated by:

$$\begin{aligned}\oint ^{a}+\oint ^{id}+\sum _{j=\kappa _{a}+1}^{\kappa }\oint _{j}\end{aligned}$$

Proof

From Equation 10, it is known that

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa } (\epsilon ^{a}_{j}\varPi _{j}(t) + \epsilon ^{id}_{j}(1-\varPi _{j}(t))) \end{aligned}$$

(28)

By expanding, we get

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa } (\epsilon ^{a}_{j}\varPi _{j}(t) + \epsilon ^{id}_{j} - \epsilon ^{id}_{j}\varPi _{j}(t)) \end{aligned}$$

(29)

Next, we characterized a new function ’$\oint$’, for the Difference Value of Energy Consumption, whose value is always positive. Since it is evident that the active energy consumption of a cloud server will always be higher than the value of its idle energy consumption.

$$\begin{aligned} \oint =\epsilon ^{a}-\epsilon ^{id} \end{aligned}$$

(30)

By substituting Eq. 30 in the Eq. 29, we obtain,

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa }\varPi _{j}(t)\oint _{j} + \sum _{j=1}^{\kappa }\epsilon ^{id}_{j} \end{aligned}$$

(31)

Here, we consider the $\kappa _{a}$ denotes the active servers in the ideal condition. Thus, it is evident that, $1\le \kappa _{a}\le \kappa$. We can note that it can not be implied that only $\kappa _{a}$ cloud servers would be in active mode in the practical run-time of our approach. Using $\kappa _{a}$ in the Equation 31,

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa }\varPi _{j}(t)\oint _{j} + \sum _{j=1}^{\kappa _{a}}\epsilon ^{id}_{j} + \sum _{j=\kappa _{a}+1}^{\kappa }\epsilon ^{id}_{j} \end{aligned}$$

(32)

Let’s introduce a new term $\sum _{j=1}^{\kappa _{a}}\epsilon ^{a}_{j}$ for Equation 32,

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa }\varPi _{j}(t)\oint _{j} + \sum _{j=1}^{\kappa _{a}}\epsilon ^{id}_{j} - \sum _{j=1}^{\kappa _{a}}\epsilon ^{a}_{j} + \sum _{j=\kappa _{a}+1}^{\kappa }\epsilon ^{id}_{j} + \sum _{j=1}^{\kappa _{a}}\epsilon ^{a}_{j} \end{aligned}$$

(33)

By applying Eq. 30 and Eq. 26 in the Eq. 33, we obtain:

$$\begin{aligned} \oint _{OTDAM}=\sum _{j=1}^{\kappa }\varPi _{j}(t)\oint _{j} - \sum _{j=1}^{\kappa _{a}}\oint _{j} + \oint _{OPT} \end{aligned}$$

(34)

As the ideal and active energy consumption costs of all the cloud servers are already known and constant. Hence, the final value of $\oint _{OTDAM}$ relies primarily on the cost of $\varPi _{j}(t)\ \forall \ {1\le j\le \kappa }$. After all, $\varPi _{j}(t)$ represents the utilization of cloud server, it is known that $\forall \ {1\le j\le \kappa }$, $0\le \varPi _{j}(t)\le 1$. Thus by applying the Eq. 27,

$$\begin{aligned} \oint _{OTDAM}\le \sum _{j=\kappa _{a}+1}^{\kappa }\oint _{j}+\oint ^{a}+\oint ^{id} \end{aligned}$$

(35)

$\square$

Corollary 1

For the proposed algorithm, the maximum value of the asymptotic approximation ratio ($\nabla _{OTDAM}$) is:

$$\begin{aligned}\frac{\oint ^{a}+\oint ^{id}+\sum _{j=\kappa _{a}+1}^{\kappa }\oint _{j}}{\oint ^{a}+\oint ^{id}}\end{aligned}$$

Proof

By applying Eq. 24 and Theorem 2, we can find:

$$\begin{aligned} \nabla _{A}=\frac{\oint _{OTDAM}}{\oint _{OPT}}\le 1+\frac{\sum _{j=\kappa _{a}+1}^{\kappa }\oint _{j}}{\oint ^{a}+\oint ^{id}} \end{aligned}$$

(36)

$\square$

4.5 Time complexity analysis

Theorem 3

The time complexity of Algorithm 2 is ${\mathcal {O}}(\kappa \log {}\kappa +\eta \log {}\eta +5\eta \kappa +2\kappa +4\eta )$

Proof

The second algorithm is about determining the winning bids and allocating the VM instances to the cloud servers. We consider the mergesort algorithm for sorting, whose complexity is ${\mathcal {O}}(\eta \log {}\eta )$. We have two allocation steps in the algorithm with complexities of $2\eta$ and $2\kappa$, respectively. We then have two sorting steps in the algorithm with time complexities of ${\mathcal {O}}(\eta \log {}\eta )$ and ${\mathcal {O}}(\kappa \log {}\kappa )$ separately. The for-loop in the line no. 27 contains a while-loop and another for-loop within that while-loop. We have two candidate worst cases here - first, where the value of $\lambda =\kappa$, thus making the contents of the for-loop to iterate only once and giving a time complexity of ${\mathcal {O}}(\kappa +5)$ and second, where the value of $\lambda =1$, thus making the contents of the for-loop to iterate maximum $\kappa$ times and giving a time complexity of $5\kappa +2$. We’ll consider the second case as our worst-case as its time complexity is higher. So, the time complexity of the for-loop is ${\mathcal {O}}(\eta (5\kappa +2))$ and thus, the total time complexity of Algorithm 2 is ${\mathcal {O}}(\kappa \log {}\kappa +\eta \log {}\eta +5\eta \kappa +2\kappa +4\eta )$.

$\square$

Theorem 4

The time complexity of Algorithm 3 is ${\mathcal {O}}(\eta (\log {}\eta +\log {}\kappa )+\kappa \log {}\kappa )$

Proof

The third algorithm is about calculating the payment to be given by each buyer and to be received by each seller. We assume the searching algorithm to be binary search, whose complexity is ${\mathcal {O}}(\log {}n)$. We have a sorting step in the algorithm with a time complexity of ${\mathcal {O}}(\kappa \log {}\kappa )$. The for-loop in the third line contains another for-loop. The worst case is where the VMs are allocated such that every server becomes a winning seller, and every buyer becomes a winning buyer, thus making the for-loop in the third line iterate $\kappa$ times, and the for-loop in the fourth line iterate a total of $\eta$ times. The complexity of binary search (in sixth and eleventh statements) is ${\mathcal {O}}(\log {}n)$. So, the total complexity of the statements within the loops is ${\mathcal {O}}(\log {}\eta +\log {}\kappa )$, and the time complexity of whole algorithm is ${\mathcal {O}}(\eta (\log {}\eta +\log {}\kappa )+\kappa \log {}\kappa )$. $\square$

Theorem 5

The time complexity of Algorithm 1 is ${\mathcal {O}}(2\eta \log {}\eta +\eta \log {}\kappa +5\eta \kappa ).$

Proof

Algorithm 1 time complexity can be obtained as the sum of complexities of Algorithm 2 and Algorithm 3. Considering that $\eta>>\kappa$, from Theorem 3 and Theorem 4, we can calculate the overall time complexity of Algorithm 1 and can be written as:

$$\begin{aligned}&{\mathcal {O}}[(2\eta +2\kappa ) + (\kappa \log {}\kappa +\eta \log {}\eta +5\eta \kappa +2\eta ) \nonumber \\&\quad + (\eta \log {}\eta +\kappa \log {}\kappa +\eta \log {}\kappa )]\nonumber \\&\quad \approx {\mathcal {O}}[2\eta \log {}\eta +\eta \log {}\kappa +5\eta \kappa ] \end{aligned}$$

(37)

$\square$

4.6 Properties of OTDAM

Theorem 6

OTDAM follows individual rationality.

Proof

Any auction design is individually rational if and only if the buyer’s payment is less than or equal to its bidding value, and the seller also receives greater than or equal to its asking value.

As, it is known that $\forall$ $B_{i}$ in B, there can be two cases:

• 1. $B_{i}$ is not a winning buyer. Thus, it is not necessary to calculate its rationality.

• 2. $B_{i}$ is a winning buyer. We have two more situations as described in Algorithm 3 -

  • (a) $B_{i}$ is not the last buyer of $\varTheta$. It is known that $\alpha _{j}\le \beta _{i}$, and it is also a major criteria for $B_{i}$ to be declared as a winning bid. Also, it is known that $\beta _{\delta +1}\le \beta _{i}$ since the buyers in $\varTheta$ are positioned in non-increasing sequence of their bid-per-instance value. Hence, $\chi _{i}\le \varUpsilon _{i}\beta _{i}$.

  • (b) $B_{i}$ is the last buyer of $\varTheta$. $\chi _{i}=\varUpsilon _{i}\beta _{i}$.

As, we know that $\forall$ $S_{j}$ in S, there can be two cases:

• 1. $S_{j}$ is not a winning seller. Hence, it is not necessary to calculate its rationality.

• 2. $S_{j}$ is a winning seller. Thus, we have two more cases as described in Algorithm 3 -

  • (a) $S_{j}$ is not the last seller of $A^{\alpha }$. As, it is known that $\beta _{i}\ge \alpha _{j}$, and it is also a major criteria for $B_{i}$ to be declared as a winning bid. Also, it is known that $\alpha _{\omega +1}\ge \alpha _{j}$ since the sellers in $\varOmega$ are positioned in non-decreasing order based on their ask-per-instance value. Thus, $\chi _{j}\ge (\psi _{j}-\phi _{j})\alpha _{j}$.

  • (b) $S_{j}$ is the last buyer of $\varOmega$. $\chi _{j}=(\psi _{j}-\phi _{j})\alpha _{j}$.

The Individual Rationality of OTDAM has also been illustrated in Figs. 4 and 5. $\square$

Lemma 1

OTDAM is always truthful for buyers.

Proof

Let $\beta _{i}$ be the true valuation bid of the buyer $B_{i}$ and $\beta '_{i}$ be the false bid buyer $B_{i}$ places in the greed of a better utility. Now, we have two possible cases

• Case 1 : $\beta '_{i}<\beta _{i}$ If the buyer places a bid that is less than its true valuation, it might jeopardize its chances of allocating resources at a more energy-efficient server. Also, in some cases, there is a chance that the buyer might not be allocated at all since the buyer won’t know the asking prices as this is a sealed-bid auction. So, the buyer might suffer a loss if it places a bid less than its true valuation.

• Case 2 : $\beta '_{i}>\beta _{i}$ If the buyer places a bid whose value is greater than its true valuation, it might end up being allocated to a higher-ask seller, thus reducing its utility. Also, the payment mechanism is such that if the buyer’s bid is significantly higher than its true valuation, it might have to pay an amount greater than its valuation, which is undesirable. So, a buyer stands a huge chance of suffering a loss if it places a bid higher than its true valuation.

So, in either of the two cases, the best policy for a buyer is to place a bid equal to its true valuation. $\square$

Lemma 2

OTDAM is always truthful for sellers.

Proof

Let $\alpha _{j}$ be the true cost ask of the buyer $S_{j}$ and $\alpha '_{j}$ be the false ask seller $S_{j}$ places in the greed of a better utility. Now, we have two possible cases

• Case 1 : $\alpha '_{j}<\alpha _{j}$ If the seller places an ask less than its true cost, it might end up jeopardizing its chances of winning the auction since the seller won’t know the bidding prices as this is a sealed-bid auction. Also, the payment mechanism is such that if the seller’s ask is significantly lower than its true cost, it might get paid less than its cost, which is undesirable. So, a seller stands a huge chance of suffering a loss if it places an ask lower than its true cost.

• Case 2 : $\alpha '_{j}>\alpha _{j}$ If the seller places an ask higher than its true cost, there is a chance that no buyer might be able to match its asking price, and the seller might lose the auction.

So, in either of the two cases, the best strategy for a seller is to place an ask equal to its true cost. $\square$

Theorem 7

OTDAM is truthful.

Proof

By applying Lemma 1 and Lemma 2 simultaneously prove that OTDAM is truthful. The truthfulness of OTDAM has also been illustrated in Fig. 6. $\square$

Theorem 8

OTDAM is also economically efficient.

Proof

OTDAM always prefers a buyer bidding high bid-per-instance. Hence the costly resources of sellers will always be given to those buyers who are willing to pay more. Thus, OTDAM is economically-efficient. $\square$

5 Performance evaluation

We have conducted a set of simulation experiments to examine the efficacy of OTDAM while allocating resources in IaaS clouds. Subsection 5.1 illustrates the experimental environment and the benchmark. The evaluation methods are explained in Subsection 5.2. We also compare the performance of these approaches and analyze the experimental results in Subsection 5.3.

5.1 Experimental environment

To investigate the overall performance of proposed auction mechanisms, we have applied trace-driven simulations, utilizing Google cluster-usage data [46]. The dataset captures the activity of a production cluster comprising a total of 12,583 machines with a heterogeneous configuration for a period of 29 days. Machines are categorized into three different classes and ten distinct architectures with available CPU and memory capacity, as shown in Table 12. In the table, the memory and CPU units are linearly scaled in such a way that the maximum resource capacity is 1.

Table 12 Configuration of machines in the Google cluster

To simulate a cloud data center with servers of heterogeneous resource capacities, we have configured the hosts based on the distribution of cloud server configurations reported in the Google cluster data, also shown in Table 13. The hardware specification and energy consumption values for the hosts were accumulated from the well-known SPECpower [47] benchmarks. Since the disk configurations of servers are not available in the Google dataset, we randomly set the capacity of disk storage for servers within [320, 800](in GB).

Table 13 Host characteristics for Google’s cloud

Resource tables are broadly classified into three major categories: Machines, Jobs and Tasks, and CPU Usage. Each class has one or more than one table containing multiple features. The dataset comprises approximately 672,074 incoming real-time jobs, and an individual user creates each job out of total 925 users. In Google cluster data, each job consists of multiple tasks that unite to approximately 24,281,242 different tasks. Each task is having three types of resource requirements, i.e., RAM, CPU, and disk. The usage of every type of resource is collected at five minutes intervals. It also provides the arrival time for each job and comprises task duration, i.e., start-time and finish-time of each task present in a particular job. The dataset illustrates that most of the tasks only run for short durations. There are also some of the long-running tasks that would yield some weeks to finish their execution. Our experiment comprises of six VM types of Amazon’s instance classes, as shown in Table 14.

Table 14 Types and characteristics of Amazon EC2 instances

Table 15 Hardware and software specification of HPC Cluster

Each VM instances is randomly allocated a workload trace from the data set, ensuring the demands extracted from the cluster traces at time t should be less than the total capacity of VM. We set the start-time and finish-time of VM usage based on the corresponding tasks’ latest time slot. The value of T is set to 3000 and each of duration 10 seconds. The assumption of other values for the following variables lie in range: $\beta$ $\in$ [4.0,14.0), $\alpha$ $\in$ [2.0,20.0), $\tau$ $\in$ [3,10) (milliseconds). We have implemented the auction mechanism in the python programming language using an HPC cluster. The cluster contains one management server, and four compute servers. Table 15 shows the details, including hardware and software specifications. Each compute node is connected with the InfiniBand network (56 Gbps) and ethernet to the management server.

5.2 Evaluation methods

To analyze the performance of OTDAM, we compare its experimental results with three other existing double auction approaches: TASC [11], TDAM [42], and ICAM [43]. Each experiment is run over 100 times and averaged as the final result. The auction mechanisms are evaluated by using the following criterion: (i) Successful trades, (ii) Revenue, (iii) Energy-performance, (iv) Time-slots, (v) Volatility, and (vi) Running time. We have also discussed a theoretical comparison among auction mechanisms in Table 16.

5.3 Analysis of results

We first investigate the impact of different performance parameters and analyze the results. Afterward, we discuss the theoretical comparison of OTDAM results with three existing double auction-based mechanisms.

Fig. 8

Comparison of successful trades

Fig. 9

Analysis of revenue maximization

Fig. 10

Energy-performance analysis

Fig. 11

Time slots versus Successful trades

Fig. 12

Volatility analysis

Fig. 13

Running time analysis

• Number of successful trades In Fig. 8, we examine the successful trades of OTDAM and compare its performance with the other three approaches. In this experiment, we have varied the buyers (users) from 0 to 14,000 and sellers (servers) from 0 to 7000 by increasing 2000 numbers of buyers and 1000 sellers respectively at each step. From Fig. 8, it can be observed that the plot for successful trades is a linearly increasing graph for all the four algorithms used for comparison. TASC wins over by giving maximum successful trades with an increase of servers and VMs, as illustrated in Fig. 8. OTDAM initially provides a good amount of successful trades in comparison with TDAM and ICAM. Still, as the number of VMs and servers increase respectively in a particular slot, it becomes difficult for servers to allocate more VMs within that specific period. For example, in Fig. 8, it is depicted that when the number of buyers is 14,000, the OTDAM realizes approximately 11,000 successful trades, and the TASC achieves around 15,500 successful trades. The interesting thing in our proposed auction approach is, it compares the bid and ask values at each period, and the winning seller obtains a final payment not less than its ask price, and the buyers are not charged more than its bid value.

• Revenue maximization In Fig. 9, we illustrate the revenue generated by OTDAM and compare its performance with the other approaches. To verify its performance, we pick a fixed number of buyers and sellers. From Fig. 9, it can be observed that among the four algorithms, the revenue generation is maximum for OTDAM after a certain point, i.e., 7000 buyers. The point where the revenue generated is increasing when buyers and sellers are in the range of thousands, and it takes time in minutes to complete the execution, as shown in Fig. 9. It is also observed that, the TASC results in worst performance than TDAM. The reason behind the maximum revenue generation is the seller desirability factor since the VM with the highest bid is allocated first to the servers with the highest ask-per-resource value. Thus, it maximizes the VM allocation by always placing the costly resources of the seller to a buyer having a high valuation, and being able to pay. As a result, it maximizes the overall revenue for a larger set of buyers and sellers, respectively.

• Energy-performance analysis Figure 10 compares the energy performance of the four approaches. For this experiment, we have varied the buyers from 200 to 1000 and sellers from 100 to 500 by increasing 200 numbers of buyers and 100 numbers of sellers respectively at each step. From Fig. 10, it can be identified that the energy consumption of the VMs is increasing linearly for the ICAM algorithm, which is the only one under-performing than OTDAM. But considering the general case, the total energy consumption of OTDAM is minimum for maximum successful trades. The reason behind the minimum energy consumption is the seller-desirability value, which always allocates the VM instances to the cloud server having the least active energy consumption.

• Time slot analysis Figure 11 examines the number of successful trades each algorithm experiences with the increase in time slots. For the time-slots versus successful trade analysis, we have varied the number of time-slots from 10 to 40 by increasing five numbers of time-slots respectively at each step. From Fig. 11, it can be observed that the OTDAM experiences a drop initially, but after 15-time slots, it maximizes the trades with an increase in time slots. It can also be noticed that when the time-slots are 35, the OTDAM achieves more than 1000 successful trades while the TDAM gives fewer successful trades.

• Volatility analysis Figure 12 investigates the impact of market volatility on the allocation efficiency by taking out the standard deviation of the valid price schedules. For this experiment, we have ranged from 10 to 35 time-slots and the number of trades from 0 to 1400 by increasing five numbers of time-slots and 200 numbers of trades respectively at each step. From Fig. 12, it can be observed that when the market is at low volatility trades are reasonably low. For example, when the trades are 200 at 10th time-slots, the volatility value is around 0.63 for OTDAM. As the volatility increases, the allocation efficiency becomes better. For example, at 23rd time-slots, OTDAM achieves approximately 1.5 volatility value, which is higher than the TASC approach, which produces only 1.0 volatility. However, the TDAM and ICAM approaches perform well and experience stable changes.

• Running-time analysis Figure 13 compares the running time of all four algorithms. For the running-time analysis, we have varied the buyers from 0 to 14,000 and sellers from 0 to 7000 by increasing 2000 numbers of buyers and 1000 numbers of sellers respectively at each step. From Fig. 13, it can be identified that all the algorithms have a related functioning for a small amount of VM instances and servers. Still, as the number increases, the TASC mechanism takes more time to complete its allocation, i.e., taking more than 600 seconds to satisfy 14000 number of buyers. The OTDAM clearly shows its optimized technique saving upon running time even with a large set of of buyers and sellers, as depic in Fig. 13. For instance, when the buyers are 14,000, OTDAM takes less than 400 seconds to fulfill the request of VMs, which is much less than TASC and TDAM taking over 600 and 400 seconds, respectively. It can also be identified that the running-time of OTDAM increases with respect to the rise in the number of participating buyers and sellers separately. Thus, OTDAM is subjected to a polynomial computation time concerning the increasing number of buyers and sellers.

• Theoretical comparison In Table 16, we present a theoretical comparison between the four auction approaches: TASC, TDAM, ICAM, and OTDAM. The auction mechanisms are evaluated using the following important criterion: (i) Auction type, (ii) Truthfulness for buyers, (iii) Truthfulness for sellers, (iv) Individual rationality, (v) Buyers utility, and (vi) Computational complexity. In summary, OTDAM promises truthfulness of buyers and sellers considering the utility cannot be enhanced by bidding or asking untruthfully. It also guarantees individual rationality and higher utility values for buyers while taking polynomial computation time corresponding to the more significant number of buyers and sellers.

Table 16 Comparison between TASC, TDAM, ICAM, and OTDAM

6 Conclusion

This paper studies the limitations of existing auction techniques for dynamic resource allocation in IaaS clouds. We have observed that the current auction techniques are mainly one-sided or offline without considering the dynamics of an elastic model in a time-variant environment. To address the issues of the current auction approaches of VM allocation, we have devised an online truthful double auction approach ’OTDAM’ with the unique features of truthfulness, elasticity, heterogeneity, and multi-objective trade-offs for dynamic resource allocation in IaaS clouds. OTDAM has two main algorithms first one is weighted bipartite matching based winning bids determination, and the second is a VCG based algorithm for payment calculation. OTDAM can impressively distribute the cloud resources between the cloud users to meet their instance demands while ensuring the desirable auction features such as individual rationality, truthfulness, economic-efficiency, and polynomial-time computational complexity. Further, we have provided rigorous proofs on the OTDAM auction properties and validate the analysis with extensive trace-driven simulations showing improvements over performance, revenue, energy consumption, and running-time.

As a future study, we plan to investigate the emerging constraint challenges in distributed data centers, such as anti-affinity, scalability, network resource utilization, and availability etc., and apply these constraints to online resource provisioning algorithms to evaluate its performance in a distributed environment.