Adaptive mechanism design and game theoretic analysis of auction-driven dynamic spectrum access in cognitive radio networks

Abstract

This paper studies the auction-driven dynamic spectrum access in cognitive radio networks with heterogeneous secondary users, who have different risk attitudes. First, a game theoretic framework is established for auction-driven dynamic spectrum access in cognitive radio networks. The utility functions and bidding strategies of heterogeneous secondary users are defined, and the parameterized auction mechanisms of primary user are also introduced. Then, we formulate the auction-driven dynamic spectrum access problem as a finite discrete game with a mixed- or pure-strategy Nash equilibrium solution. We study the existence and uniqueness properties of the pure-strategy Nash equilibrium in the defined game. Next, we propose a distributed learning automata algorithm (DLA) to attain the Nash equilibrium of the defined game with limited feedback. The adaptive mechanism design is realized in the updating procedure of our DLA algorithm. We further prove that our DLA algorithm converges to a Nash equilibrium of the defined game. Finally, simulation results show that our DLA algorithm is efficient and outperforms the dynamic spectrum access schemes with fixed auction mechanism.

1 Introduction

Cognitive radio aims to improve the utilization of radio electromagnetic spectrum, which is scarce but often underutilized [1]. To realize efficient spectrum utilization, various spectrum management techniques have been proposed for cognitive radio networks, and the auction-based spectrum management has recently attracted wide attention [2, 3].

Existing work mainly focused on using game theory to analyze the behaviors of the secondary users under a certain auction mechanism. Furthermore, the secondary users are usually assumed to be symmetric and homogeneous. Symmetry and homogeneity of secondary users are, however, generally not realistic in practical cognitive radio networks. In practice, secondary users usually have different risk attitudes in the auction, e.g., some secondary users are with high channel gain or urgent message and thus may tend to be risk-seeking, some secondary users are with non-urgent messages or low channel gain and thus may tend to be risk-averse, and the others may be regarded as risk-neutral buyers between risk-seeking and risk-averse buyers [4, 5].

Byde, 2003 [4] and Maskin and Riley, 1984 [6] have studied the auction with asymmetric bidders and have shown that different auction mechanisms would result in different outcomes. In other words, a fixed auction mechanism may not be optimal for the auctioneer in all circumstances [7, 8]. In order to obtain higher revenue, auction mechanism should be adaptively designed by the auctioneer according to the types of the secondary users. Therefore, for the cognitive radio networks with heterogeneous (multi-type) secondary users, it is of profound importance to study the auction-driven dynamic spectrum access with adaptive auction mechanism design.

Mechanism design [7] is applied to dynamic spectrum access in cognitive radio networks [9]. A generic parameterized auction mechanism design is presented in [9], but only two specific mechanisms are considered to suppress cheating and collusive behavior of selfish users, which contains a class of mechanisms for heterogeneous buyers. In the adaptive auction mechanism designs in [4, 10, 11], the risk attitudes of the buyers are not taken into account. Whereas in [4] evolutionary game theory is applied to study the auction mechanism design, in our proposed scheme, distributed learning automata are applied to auction mechanism design in cognitive radio networks and the dynamic spectrum access is realized jointly for secondary users. It is shown in [12] that replicator dynamics based on evolutionary game theory and the learning automata [13] are quite similar and are actually equivalent in some circumstances. However, learning automata are computationally simple and efficient and thus are more appropriate in designing practical distributed algorithm with limited information. Teng et al., 2011 [14] presents a dynamic spectrum sharing through a double auction mechanism, and [15] designs a truthful online spectrum auction for efficient spectrum allocation. However, neither auction model has considered heterogeneous secondary users.

Shen et al., 2011 [16] studies dynamic spectrum auction in wireless networks based on coexistent matrix, and [17] studies network pricing of private wireless access points (AP) through a location-dependent multi-AP reverse auction. Wang et al., 2012 [18] designs an efficient scheme of spectrum auction game for near-optimal spectrum allocation. In [19], an auction-based game theoretic approach is used for relay power allocation in wireless networks. However, in all these auction models, neither heterogeneous buyers with different risk attitudes nor adaptive auction mechanism design has been considered.

In this paper, we study cognitive radio networks with multiple secondary users that have different risk attitudes (i.e., heterogeneous buyers). The secondary users compete to access an idle channel under an auction simultaneously. Moreover, the primary user, i.e., the auctioneer, is able to apply multiple sealed-bid auction mechanisms that are adaptively designed. In practice, users would not have exactly full information of the network. Therefore, the aim of this paper is to address the following questions: how the auctioneer should design the auction mechanism adaptively to maximize the utility and how the secondary users should choose their best bidding strategies with limited or local information.

For this purpose, we formulate the auction-driven spectrum access problem as a discrete game and define appropriate utility functions for both the secondary users and the primary user. We then study the existence and uniqueness properties of the Nash equilibrium solution of the defined game. Given that the primary user and the secondary users would not have full information on cognitive radio networks, centralized algorithm is impractical. Therefore, we propose a practical distributed learning automata algorithm to attain the Nash equilibrium with only limited feedback information.

The remainder of the paper is organized as follows. In Section 2, the system model of cognitive radio network is presented. In Section 3, a game theoretic framework is established for auction-driven dynamic spectrum access in cognitive radio networks. Specifically, bidding strategies and utility functions are defined for the secondary users, and a generic parameterized auction mechanism and the utility function are defined for the primary user. Then, the auction-driven dynamic spectrum access in cognitive radio networks is formulated as a finite discrete game. In Section 4, a distributed learning automata (DLA) algorithm is proposed for evolving the defined mixed-strategy game to attain the Nash equilibrium solution to the auction-driven dynamic spectrum access. The adaptive mechanism design is realized in the updating procedure of our DLA algorithm, and the convergence of our DLA algorithm is proved. In Section 5, simulation results are presented for auction-driven dynamic spectrum access in cognitive radio networks. Finally, conclusions are drawn in Section 6.

2 System model of cognitive radio networks

The system model of the cognitive radio network, as depicted in Figure 1, consists of one primary user and multiple secondary users coexisting with the primary user. At a specific instant, we assume that there are n different secondary users simultaneously sensing the idle spectrum (i.e., channel) of the primary user. From the resource market point of view, the channel are the goods to be auctioned, the primary user is the seller or auctioneer, and the secondary users are the buyers or bidders who are competing to win the goods. For the sake of fairness and maximum revenue, the primary user shall adaptively design an auction mechanism, under which the secondary users compete for the channel. We assume that the secondary users choose their own bidding strategies after evaluating the channel conditions and other necessary information fed back from the primary user. The primary user has multiple, say M, auction mechanisms to apply. The secondary users employ the same set of bidding strategies.

Figure 1

The system model of auction-driven dynamic spectrum access in a cognitive radio network.

We assume that in each fixed duration, the primary user would organize the auction for the secondary users to compete under and announces the available channels at the beginning of every auction. Suppose that each channel allows only one secondary user to access at an instance. Whenever a secondary user wins the bid, a certain portion, proportional to its bid, would be registered. After the data transmission time, the secondary user who has successfully accessed the channel shall pay the price at the end of the frame.

The channel gain between secondary user i and the primary user is h _i . $\forall i\in \mathcal{N}$, h _i is assumed to be invariant over duration T, where $\mathcal{N}=\{1,2,\cdots ,n\}$ is the index set of the secondary users. We should point it out that different channel models will not affect the theoretic analysis and results of this paper.

For convenience of reference, Table 1 lists the notations used in this paper.

Table 1 Notations

3 Strategy game formulation of auction-driven dynamic spectrum access

3.1 Bidding strategies and utility functions of secondary users

Let b _i denote the bidding strategy of secondary user i and b _i ∈B _i . $B_i=\{b_{i1},\cdots ,b_{\mathit{\text{ik}}},\cdots ,b_{{\mathit{\text{iK}}}_i}\}$ is an ordered discrete finite set of bidding strategies of secondary user i, where K _i is the cardinality of B _i , K _i ≥2, b _{i k} is the k th bidding strategy in B _i , and $b_{i1}<\cdots <b_{{\mathit{\text{iK}}}_i}$. For each secondary user i, b_{i 1} represents the null bid and without loss of generality, is simply normalized to be zero; b_{i 2} denotes the lowest non-zero admissible bid, i.e., the reserve price, and it is assumed that b_{i 2}≥1. For the sake of simplicity, assume that $\forall i,j\in \mathcal{N}$, b_{i 2}=b_{j 2}. Moreover, the auction should be replayed if all secondary users choose the null bid.

Assume that the secondary users are only interested in their own awards and payments. In general, all secondary users attempt to win the channel with a low cost. The utility reward to secondary user i who wins the channel at bid b _i is a discrete function, denoted as u _i (v _i (C _i ,b _i )), where v _i is defined as an increasing function of the channel capacity (C _i ) of secondary user i, but a decreasing function of b _i . u _i (v _i ) reflects the value of the goods (i.e., channel) to secondary user i. C _i is defined as

$$C_i=W\underset{2}{log}\left(1+\underset{\text{SNR}}{\underbrace{\frac{{\rho }_i{h}_i}{{\sigma }^2}}}\right),$$

(1)

where ρ _i is the transmit power of secondary user i, W is the bandwidth of the channel, σ² is the mean channel noise power, and SNR denotes signal-to-noise ratio.

In practice, different secondary users usually have different requirements on services. Some secondary users may have urgent messages, and some may not have. Consequently, secondary users may fall into heterogeneous rather than the same type. Since secondary users woould not have full information of the system, they may exhibit different risk attitudes when they face uncertain utilities. For example, secondary users with real-time services, or high SNR, or urgent messages, may be aggressive about the risks and try their best to access the channel, so they are risk-seeking bidders [4]. However, secondary users with non-real-time services, or low SNR, or non-urgent messages, may not be so aggressive and thus are risk-averse and in other cases, secondary users are risk-neural bidders [4]. The characterization of the heterogeneous secondary users and the risk attitude is described in Table 2.

Table 2 Characterization of heterogeneous secondary users

To mathematically characterize the aforementioned three types of risk attitudes, i.e., risk-seeking, risk-averse, risk-neural, we adopt the von Neumann-Morgenstern utility function (i.e., expected utility function) to differentiate the risk attitudes of secondary users. Specifically, if the expected utility E(u _i (v _i )) of secondary user i is larger (or smaller) than the utility u _i (E(v _i )), then secondary user i prefers not to or prefers to take the risk presented by the uncertainty, and thus, secondary user i is risk-averse (or risk-seeking). According to microeconomics such as textbook [7], utility functions of the three types of risk attitudes should follow the curves as illustrated conceptually in Figure 2.

Figure 2

Illustration of utility functions for risk-seeking, risk-neutral, and risk-averse secondary users.

As mentioned above, heterogeneous secondary users usually have different risk attitudes u _i ,i=1,2,⋯,n. In our work, u _i is defined as

$$u_i=\left\{\begin{array}{ll}\frac{1}{{\beta }_i}\left(e^{{\beta }_i{v}_i}-1\right),& \text{if}{\beta }_i\ne 0\\ v_i,& \text{if}{\beta }_i=0\end{array}\right.$$

(2)

$$v_i\left(C_i,b_i\right)={\theta }_i{C}_i-{\widehat{b}}_i,$$

(3)

where β _i is the risk attitude parameter of secondary user i, ${\widehat{b}}_i$ is the price that the winning secondary user i should pay to the primary user, and θ _i >0 is the parameter controlling the sensitivity of the channel quality for secondary user i and keeping the term of C _i consistent with the unit of b _i .

As shown in Figure 2, when β _i >0, it can be seen that the expected utility E(u _i (v _i )) of secondary user i is smaller than its real utility u _i (E(v _i )), which means that secondary user i tends to take the risk presented by the uncertainty; so, it is risk-seeking. When β _i <0, the expected utility E(u _i (v _i )) of secondary user i is smaller than its real utility u _i (E(v _i )), which means that secondary user i tends not to take the risk presented by the uncertainty; so, it is risk-averse. When β _i =0, the expected utility E(u _i (v _i )) of secondary user i is equal to its real utility u _i (E(v _i )), which means that it is risk-neutral.

In practice, the value of β _i depends on the type of secondary user i. In our work, as shown in Table 2, β _i >0 corresponds to a risk-seeking secondary user i who has urgent message or real-time service and is rich, β _i <0 to a risk-averse secondary user i who has neither urgent message nor real-time service and is poor of money, and β _i =0 to a risk-neutral secondary user i who has no urgent message and is of middle class.

Moreover, we assume that if all secondary users choose the same bid, no secondary users are allowed to access the channel and the auction should be replayed. In such a situation, the utility of each secondary user and the revenue of the primary user are equal to zero.

In summary, the utility function of the secondary users can be expressed as below

$$U_i=\left\{\begin{array}{ll}{u}_i,& \mathit{\text{winning}}\\ 0,& \text{otherwise}\end{array}\right.,$$

(4)

where u _i is given in (2).

Note that this is a generic model of the heterogeneous secondary users. All secondary users being of the same type can be regarded as a special case of our proposed model.

3.2 Auction mechanism design and utility function of primary user

There are numerous kinds of auction mechanisms, such as progressive auctions, Dutch or British auctions, sealed-bid auctions, and discriminatory auctions. In our work, we assume that the auction mechanisms are parameterized into a finite discrete set.

Let’s take sealed-bid auction as an example. In a seal-bid auction, a goods is put up for sale, and each potential buyer submits a bid to the auctioneer; the auctioneer chooses a winner and receives the payment from the winning bidder. In most variants of sealed-bid auction, the goods is awarded to the buyer who submits the highest bid, and only the winner pays. There are two standard sealed-bid auctions. In first-price auction, the winner’s payment is its bid; in second-price auction, the winner’s payment is the second highest bid. In this paper, instead of relying on a particular auction, we parameterize a class of auction mechanisms including both standard sealed-bid auctions and non-standard ones.

In [4], a concept of vector-price auction is designed as follows.

Definition 1.

Let $\omega =\left({\omega }_1,\cdots ,{\omega }_{\widehat{n}}\right)$ be a vector of $\widehat{n}$ real numbers. A ω-price auction is a sealed-bid auction in which the highest bidder, denoted as i^∗, wins the goods and pays a calculated price as follows

$${\widehat{b}}_{i^{\ast }}=\frac{{\sum }_{i=1}^n{\omega }_i{\text{bid}}_i}{{\sum }_{i=1}^n{\omega }_i},$$

(5)

where n is the minimum between $\widehat{n}$ and the number of bidders, and bid₁, bid₂,… are the bids ordered from the highest to the lowest.

By using a special two-dimensional sub-space of ω-price auctions, namely ω=(1−ω₂,ω₂) with 0≤ω₂≤1, we can already characterize a class of auction mechanisms. In particular, ω₂=0 corresponds to a standard first-price auction, and ω₂=1 corresponds to a standard second-price auction; all other values of ω₂ correspond to non-standard auctions. In this paper, we assume that the value of ω₂ is chosen from a finite set Ω. Let M be the cardinality of Ω. Then, there are M mechanisms available for the primary user to apply. Thus, the auctioneer can adjust parameters in response to the results observed from the previous auctions.

The primary user plays the role of an auctioneer, which involves organizing the auction process and designing an auction mechanism out of all different kinds of auction mechanisms that is able to achieve the maximum expected utility. Denote $\Omega =\left\{{\omega }_2^1,\cdots ,{\omega }_2^m,\cdots ,{\omega }_2^M\right\}$, where ${\omega }_2^m$ is the m th element in Ω. Then, whenever ${\omega }_2^m$ is given, the auction mechanism is actually specified. In other words, $\forall m\in \{1,\cdots ,M\},{\omega }_2^m$ corresponds to $\left(1-{\omega }_2^m,{\omega }_2^m\right)$-price sealed-bid auction. Then, choosing an auction mechanism becomes simply choosing ${\omega }_2^m$ from Ω.

With a specific auction mechanism, the primary user receives the revenue from the payment of the winning bidder. Moreover, to guarantee the efficiency of the system, the capacity of the system should also be taken into consideration. Let r denote the utility function (i.e., the revenue) of the primary user, then

$$r\left({\omega }_2^m\right)={\widehat{b}}_{i^{\ast }}$$

(6)

$${\widehat{b}}_{i^{\ast }}=\left(1-{\omega }_2^m\right){\text{bid}}_1+{\omega }_2^m{\text{bid}}_2,$$

(7)

where i^∗ denotes the secondary user who wins the auction. For example, if the primary user designs (0.8,0.2)-price sealed-bid auction mechanism, the bid of the winner is bid₁, and the second highest bid is bid₂; the utility function of the primary user is then given as 0.8∗bid₁+0.2∗bid₂.

3.3 Game theoretic formulation of auction-driven dynamic spectrum access

3.3.1 Game theoretic setting

By viewing the channel as a goods in the resource market, the primary user attempts to design the best mechanism to maximize its utility, while each secondary user attempts to choose the best bidding strategy to win the auction and maximize its utility. Mathematically, for the primary user, we have

$$\underset{{\omega }_2\in \Omega }{\text{max}}r,$$

(8)

and for the secondary users, we have

$$\underset{b_i\in B_i}{\text{max}}{U}_i,\forall i\in \mathcal{N.}$$

(9)

From a game theoretic point of view, the primary user and the secondary users constitute the players; the auction mechanism and the bidding strategies constitute the pure strategies of the players. The utilities of the secondary users and the utility of the primary user constitute the payoffs of the players. Then, the interaction between the primary user and the secondary users follows the framework of a finite discrete game, which can be defined as follows:

$$\mathcal{G}=[\underset{\mathsf{\text{Player set}}}{\underbrace{\{\mathcal{N},\text{primary user}\}}},\underset{\mathsf{\text{Pure-strategy set}}}{\underbrace{\{{\left\{B_i\right\}}_{i\in \mathcal{N}},\Omega \}}},\underset{\mathsf{\text{Payoff functions}}}{\underbrace{\{{\left\{U_i\right\}}_{i\in \mathcal{N}},r\}}}].$$

(10)

3.3.2 Analysis of pure-strategy Nash Equilibrium

It is known from Nash [7] that every finite game in a strategic form has a Nash equilibrium in mixed strategy. Thus, our defined game must have at least one Nash equilibrium in a mixed strategy. Generally speaking, it is difficult to ascertain how many Nash equilibriums has and whether the Nash equilibriums are in mixed strategy or pure strategy. However, in some cases, we can analytically characterize the existence and uniqueness of the Nash equilibrium of .

Theorem 1

If there is only one secondary user in the network $(\text{i.e.,}\mathcal{N}=\{1\left\}\right),\mathcal{G}$ possesses a pure-strategy Nash equilibrium.

Proof

From (2) and (3), it follows that if there is only one secondary user in the network (i.e., $\mathcal{N}=\left\{1\right\}$), no matter what auction mechanism the primary user has applied, the reserve price (i.e., b₁₂) will give the secondary user a larger payoff than any other bidding strategies. Therefore, the reserve price is the dominant strategy of the secondary user. Meanwhile, no matter what bidding strategy the secondary user is choosing, all auction mechanisms will give the primary user the same revenue. Consequently, we have

$$\begin{array}{l}{u}_1\left(b_{12},{\omega }_2\right)>u_1\left(b_1^{\prime },{\omega }_2^{\prime }\right),\forall b_1^{\prime }\in B_1,b_1\ne b_{12},\forall {\omega }_2,{\omega }_2^{\prime }\in \Omega \end{array}$$

(11)

and

$$\begin{array}{l}r\left(b_1,{\omega }_2\right)=r\left(b_1^{\prime },{\omega }_2^{\prime }\right),\forall b_1,b_1^{\prime }\in B_1,b_1\ne b_1^{\prime },\forall {\omega }_2,{\omega }_2^{\prime }\in \mathrm{\Omega .}\end{array}$$

(12)

According to the definition of pure-strategy Nash equilibrium [7], we can draw the conclusion that a weak pure-strategy Nash equilibrium exists in .

Furthermore, instead of employing an adaptive auction mechanism design, the primary user always applies a fixed auction mechanism; then, reduces to$Ĝ$ as below:

$$Ĝ=\left[\mathcal{N},{\left\{B_i\right\}}_{i\in \mathcal{N}},{\left\{U_i\right\}}_{i\in \mathcal{N}}\right].$$

(13)

In this case, we have the following theorem.

Theorem 2

If $B_i=B_j,\forall i,j\in \mathcal{N}$, then$Ĝ$ is an ordinarily symmetric game^a. In addition, if $K_i=K_j=2,\forall i,j\in \mathcal{N}$, then$Ĝ$ possesses a pure-strategy Nash equilibrium.

Proof.

If $B_i=B_j,\forall i,j\in \mathcal{N}$, then with the utility function defined in (4) according to the definition of the ordinarily symmetric games [20, 21], for any permutation π, U _i (b₁,⋯,b _i ,⋯,b _n ) and U_π(i)(b_π(1),⋯,b_π(i),⋯,b_π(n)) must have the same ordinal rank of the payoffs. Furthermore, from [21], it says that every two-strategy symmetric game has a pure-strategy (not necessarily symmetric, though) Nash equilibrium. Hence, the theorem is proved.

Since the pure-strategy Nash equilibrium of does not always exist in general, we should study the defined game in a mixed strategy. In a mixed-strategy form of the defined game , the expectation utility function is used as the metric of the satisfaction of each secondary user or the primary user. In the following section, we will investigate this issue by using learning automata.

4 Distributed learning automata algorithm for evolving the mixed-strategy game

4.1 Mixed-strategy game setting

In practice, the primary user usually has no prior information of the secondary users and even has no knowledge of how many secondary users are in a cognitive radio network; therefore, it may incur a large amount of overheads for the primary user to obtain the perfect information about all secondary users. Here, ‘perfect’ means complete classes of the information, e.g., risk attitude, valuations, and strategy spaces, and also means no information should have error. Likewise, the secondary users in practice are unable to have the perfect information, and they even have no knowledge of the auction mechanisms the primary user may apply. Thus, it is impractical to employ a centralized decision making process to attain the equilibrium of the game.

Based on the concept of learning automata, we propose a distributed stochastic learning algorithm to adaptively update the bidding strategies of the secondary users and the auction mechanism of the primary user in a non-cooperative manner. Stochastic learning technique has been used for price dynamics [22], discrete power control [23], multi-mode precoding strategy selection [24], and so on. Our proposed distributed algorithm involves only limited amount of feedback and is computationally simple and efficient.

To construct the learning algorithm, we consider the game in its mixed-strategy form. p _{i k} denotes the probability of secondary user i choosing bid b _{i k} , and ${\sum }_{k=1}^{K_i}{p}_{\mathit{\text{ik}}}=1$. Let ${\mathbf{p}}_i=\left(p_{i1},\cdots ,p_{\mathit{\text{ik}}},\cdots ,p_{{\mathit{\text{iK}}}_i}\right)$ be the probability distribution over B _i . Furthermore, let q=(q₁,⋯,q _m , ⋯,q _M ) be the probability distribution over Ω, where q _m denotes the probability of the primary user designing the m th auction mechanism (i.e., ${\omega }_2^m$), and ${\sum }_{m=1}^M{q}_m=1$. p _i and q are also referred to the mixed strategies of the secondary users and the primary user, respectively. Then, the expected utility gⁱ for secondary user i is given as follows:

$$\begin{array}{ll}{g}^i\left({\mathbf{p}}_1,\cdots ,{\mathbf{p}}_i,\mathbf{q}\right)& =E\left[U_i|\right.j\text{th secondary user employs}\\ \text{strategy}{\mathbf{p}}_j\text{from}{B}_j,1& \le j\le n,\text{primary user employs strategy}\\ \left.\mathbf{q}\text{from}\Omega \right]=\sum _{j_1,\cdots ,j_n}& \sum _{m=1}^M{\mu }^i\left(j_1,\cdots ,j_n,{\omega }_2^m\right)q_m\prod _{l=1}^n{p}_{{\mathit{\text{lj}}}_l},\end{array}$$

(14)

where

$$\begin{array}{l}{\mu }^i\left(j_1,\cdots ,j_n,{\omega }_2^m\right)=E\left[U_i|\right.l\text{th secondary user takes action}\\ j_l,1\le l\le n,\left.\text{primary user applies mechanism}{\omega }_2^m\right].\end{array}$$

(15)

The expected revenue of the primary user is given as

$$\bar{r}\left(\mathbf{q},{\mathbf{p}}_1,\cdots ,{\mathbf{p}}_n\right)=\sum _{m=1}^M\sum _{j_1,\cdots ,j_n}r\left({\omega }_2^m\right)q_m\prod _{l=1}^n{p}_{{\mathit{\text{lj}}}_l}.$$

(16)

4.2 Distributed learning automata algorithm for mixed-strategy game

From the learning automata point of view, the secondary users and the primary user are the automata; the bidding strategies of the secondary users and the mechanism design strategies of the primary user are the actions of the automata. q(t) is the action probability distribution of the primary user at instant t, and p _i (t) is the action probability distribution of the i th secondary user at instant t. Furthermore, the normalized utilities of the secondary users and the normalized revenue of the primary user are the reactions of the automata.

Then, we can propose a distributed learning automata (DLA) algorithm based on learning automata. In our proposed DLA algorithm, the primary user learns probability distribution q over the set of auction mechanisms (i.e., Ω) so that it can maximize its expected revenue based on imperfect information and the secondary users learn probability distributions ${\mathbf{p}}_i,\forall i\in \mathcal{N}$ over the set of bidding strategies in order to maximize their own utilities.

We assume that the primary user acts after the secondary users at each updating round, and thus, it can observe all the actions of the secondary users. That is, first, the secondary users learn and update their bidding strategies, and afterwards, the primary user learns and updates its auction mechanism. Our proposed DLA algorithm for the mixed-strategy game is elaborated in Algorithm 1, and the corresponding flowchart is illustrated in Figure 3. Distributed Learning Automata Algorithm (DLA).

Figure 3

Distributed learning automata algorithm for evolving the game in a mixed-strategy form.

In Algorithm 1, at instant t, ${\gamma }_i^s\left(t\right)$ is defined as ${\gamma }_i^s\left(t\right)={\lambda }^s{Û}_i\left(t\right)$, where ${Û}_i\left(t\right)=U_i\left(t\right)+\varphi$, ϕ≥0, and 0<λ^s<1, such that the value of ${\gamma }_i^s\left(t\right)$ lies within the interval [ 0,1). Moreover, at instant t, ${\gamma }_i^b\left(t\right)$ is defined as ${\gamma }_i^b\left(t\right)={\lambda }^b\widehat{r}\left(t\right)$, where ${\widehat{r}}_i\left(t\right)=r_i\left(t\right)+\varphi$, ϕ≥0, 0<λ^b<1, such that the value of ${\gamma }_i^b\left(t\right)$ lies within the interval [ 0,1). Since secondary users usually have no prior knowledge of their utilities, we employ a parameter self-adjusting scheme which can be stated as follows. At instant t, if $\forall i\in \mathcal{N},{Û}_i\left(t\right)>\frac{1}{{\lambda }^s}$, we set ${\lambda }^s=\frac{1}{{Û}_i\left(t\right)+\tau }$, where τ is a positive scalar; otherwise, λ^s remains unchanged. Similarly, if $\widehat{r}\left(t\right)>\frac{1}{{\lambda }^b}$, we set ${\lambda }^b=\frac{1}{r\left(t\right)+\tau }$; otherwise, λ^b remains unchanged.

4.3 Convergence of the distributed learning automata algorithm

The updating procedure of Algorithm 1 is known as a linear reward-inaction(L_R−I) [13]. Its convergence can be set out by the following theorem.

Theorem 3

Algorithm 1 converges to a Nash equilibrium of with sufficient small δ^b and δ^s for any initial points in the mixed strategies except the initial points in the pure strategies.

Proof.

We first define a function on the mixed-strategy set as follows:

$$f_{\mathit{\text{ik}}}=\left\{\begin{array}{ll}{f}_{\mathit{\text{ik}}}^s,& i\in \mathcal{N},k\in \mathcal{K}\\ f_k^b,& i=\text{primary user},k\in \mathcal{K},\end{array}\right.$$

(19)

where $i\in \{\mathcal{N},\text{primary user}\}$,

$$\mathcal{K}=\left\{\begin{array}{l}{B}_i,i\in \mathcal{N}\\ \Omega ,i=\text{primary user}\end{array}\right.$$

(20)

$$\begin{array}{l}{f}_{\mathit{\text{ik}}}^s=E\left[{Û}_i|\right.j\text{th secondary user employs strategy}\\ {\mathbf{p}}_j,i,j\in \mathcal{N},j\ne i,\text{secondary user}i\text{takes}k\text{th}\\ \text{action, and the primary user applies an auction}\\ \left.\text{mechanism}{\omega }_2^m\right]\\ =\sum _{j_1,\cdots ,j_{i-1},j_{i+1},\cdots ,j_n}\sum _{m=1}^M{\mu }^i\left(j_1,\cdots ,j_{i-1},b_{\mathit{\text{ik}}},j_{i+1},\cdots ,j_n\right)\\ q_m\prod _{l\ne i}^n{p}_{{\mathit{\text{lj}}}_l}.\end{array}$$

(21)

$$\begin{array}{l}{f}_k^b=E\left[\widehat{r}|\right.l\text{th secondary user employs strategy}{j}_l,l\in \mathcal{N},\\ \left.\text{and the primary user applies an auction mechanism}{\omega }_2^k\right]\\ =r\left({\omega }_2^k\right)\prod _{l=1}^n{p}_{{\mathit{\text{lj}}}_l}.\end{array}$$

(22)

Then, from (14) and (21), we have

$$\sum _{k=1}^{K_i}{f}_{\mathit{\text{ik}}}^s{p}_{\mathit{\text{ik}}}=g^i.$$

(23)

From (16) and (22),it follows that

$$\sum _{k=1}^M{f}_k^b{q}_k=\bar{r}.$$

(24)

Furthermore, by defining a function F as

$$F=\sum _{i\in \mathcal{N}}{g}_i+\overline{r},$$

(25)

we have $\forall i\in \mathcal{N},k\in \mathcal{K}$,

$$\frac{\mathrm{\partial F}}{\partial p_{\mathit{\text{ik}}}}=f_{\mathit{\text{ik}}}$$

(26)

and for $i=\text{primary user},k\in \mathcal{K}$,

$$\frac{\mathrm{\partial F}}{\partial q_k}=f_{\mathit{\text{ik}}}.$$

(27)

Then, according to theorem 3.3 in [13], the theorem is proved.

From Theorem 1, it is known that the convergence of the proposed algorithm is guaranteed. In addition, it is known from [13] that when a pure-strategy Nash equilibrium exists, it will be a locally asymptotically stable point of Algorithm 1.

5 Performance evaluation

In this section, we evaluate the performance of our proposed algorithm via computer simulations. Here, we set B₁=⋯=B _n =B, and K₁=⋯=K _n =K. Furthermore, it is assumed that the channel gains of the secondary users include path loss and fading effects. Mathematically, the channel gain of secondary user i is

$$h_i=\frac{\text{const}}{{\left(d_i\right)}^{\alpha }}|h_i^{\prime }{|}^2,$$

(28)

where const is a constant, α is the path loss component, $h_i^{\prime }\sim \mathbf{\text{CN}}(0,1)$, and d _i is the distance between secondary user i and the primary user. {d₁, d₂,⋯,d _n } are assumed to be randomly distributed over interval $\left[\underline{d},\overline{d}\right]$. Furthermore, it is assumed that there are five secondary users in the cognitive radio network unless otherwise stated. They are randomly distributed over interval [20 m,15 0m], i.e., $\underline{d}=20$ m and $\overline{d}=150$ m. We set values as follows: $\forall i,j\in \mathcal{N}$, B _i =B _j ={0,1,2,⋯,13}, K _i =K _j =K=14, ρ _i =ρ _j =0.01 Watts, const=0.097, α=4, W=10 Hz, σ²=5∗10⁻¹⁴ Watts, θ₁=1.5,θ₂=0.7,θ₃=0.8,θ₄=1,θ₅=1, β₁=−0.1,β₂=0,β₃=0.1,β₄=−0.1, β₅=−0.3. The set of parameterized auction mechanism of the primary user Ω=(0,0.2,0.4,0.6,0.8,1), M=6, δ^s=δ^b=0.2, τ=10, and ϕ=300.

First, we plot the evolution of probabilities q for the primary user for a specific channel realization in Figure 4. For illustration, we only plot the evolution of the choice probabilities p of bidding strategies for some selected secondary users, i.e., p₁₃,p₂₆,p₃₉,p₄₉,p₅₁₀, in Figure 5 for a specific channel realization. From these two graphs, we can see that the proposed Algorithm 1 has good convergence, and the probabilities of the auction mechanisms converge faster. This is because the number of auction mechanisms is much smaller than the total number of the bidding strategies. And, we find that the proposed Algorithm 1 converges to a mixed-strategy Nash equilibrium in this simulation. Furthermore, since we assume that the duration of the auction is fixed, we should control the runtime of Algorithm 1 so that it will not be longer than the auction duration. In practice, the adaptive stepsize scheme in [24] can be used to control the convergence speed of Algorithm 1 and limit the runtime of Algorithm 1.

Figure 4

Evolution of the probabilities of auction mechanisms (q) for the primary user.

Figure 5

Evolution of the probabilities of bidding strategies for the selected secondary users.

Then, we plot the average revenue $\sum _{i=1}^n{\widehat{b}}_i$ of the primary user in Figure 6, average sum utilities of the secondary users in Figure 7, and average sum capacities of the secondary users in Figure 8 for 500 independent channel realizations. In these figures, RAM-RB denotes the result where the primary user randomly chooses an auction mechanism and the secondary users randomly use their bidding strategies.

Figure 6

Comparison of the average revenues of the primary user.

Figure 7

Comparison of the average sum utilities of the secondary users.

Figure 8

Comparison of the average sum capacities of the secondary users.

It can be seen from Figures 6, 7, and 8 that our proposed Algorithm 1 always achieves the highest performance. Compared to RAM-RB, Algorithm 1 enables the primary user to receive higher average revenue (where the maximum improvement is about 43%). Furthermore, Algorithm 1 also enables the secondary users to achieve higher average sum utilities (where the maximum improvement is about 300%) and sum capacities (where the maximum improvement is about 32%) than the other schemes.

Furthermore, we plot the average revenues of the primary users and the average sum rates of the secondary users with different auction mechanisms in Figures 9 and 10, respectively, for a particular channel realization. The parameters in Figures 9 and 10 are the same as those in Figures 4, 5, 6, 7, and 8, except the following: β₁=1,β₂=0.5,β₃=1,β₄=1,β₅=0, and ${\theta }_i={\theta }_j=1,\forall i,j\in \mathcal{N}$. These two figures show that the proposed Algorithm 1 achieves higher average revenue than the other schemes, and the average sum rate of Algorithm 1 is the largest. Furthermore, we can see that the first-price mechanism fails to result in the highest revenue.

Figure 9

Comparison of the average revenues of the primary user with different auction mechanisms by using Algorithm 1.

Figure 10

Comparison of the average sum rates of the secondary users with different auction mechanisms by using Algorithm 1.

These results mean that the first-price auction or the second-price auction may not be optimal when the buyers have different risk attitudes. The results also indicate that our proposed algorithm can induce the secondary users to choose higher bids and achieve higher average sum utility.

Note that in Figures 9 and 10, the term 1st-price denotes the result where the primary user uses the traditional first-price auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies; the term 2nd-price denotes the result where the primary user uses the traditional second-price auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies. The term ω -price (e.g., 0.2-price) denotes the result where the primary user uses the non-standard ω-price (0.2-price) auction mechanism and the secondary users use our proposed algorithm to choose their bidding strategies.

Moreover, we plot the evolution of probabilities q for the primary user and the secondary users in Figures 11 and 12 for n=2 and in Figures 13 and 14 for n=3 over a particular channel realization. Notice that Algorithm 1 converges in all cases. However, in Figures 11 and 12, Algorithm 1 converges to a pure-strategy Nash equilibrium, while in Figures 13 and 14, it converges to a mixed-strategy Nash equilibrium. The results in Figures 11, 12, 13 and 14 have verified that the mixed-strategy Nash equilibrium always exists in our defined game model, yet a pure-strategy Nash equilibrium may not.

Figure 11

Evolution of the probabilities of auction mechanisms (q) for the primary user when n=2 .

Figure 12

Evolution of the probabilities of bidding strategies for selected secondary users when n=2 .

Figure 13

Evolution of the probabilities of auction mechanisms (q) for the primary user when n=3 .

Figure 14

Evolution of the probabilities of bidding strategies for selected secondary users when n=3 .

At last, we plot the average revenues of the secondary users with different types (i.e., different risk attitudes) in Figure 15 for 500-channel realizations. The parameters are set as below. Assume there are n=4 secondary users in the cognitive radio network. They are randomly distributed over interval [20m,150m], i.e., $\underline{d}=20$ m and $\overline{d}=150$ m. We set the values as follows: $\forall i,j\in \mathcal{N}$, B _i =B _j ={0,0.5,1,⋯,6.5}, K _i =K _j =K=14, ρ _i =ρ _j =0.05 Watts; const=0.097, α=4, W=1 Hz, σ²=5∗10⁻¹⁴ Watts, θ₁=1,θ₂=1,θ₃=1. The set of parameterized auction mechanisms of the primary user Ω=(0,0.2,0.4,0.6,0.8,1), M=6, δ^s=δ^b=0.2, τ=10, and ϕ=100.

Figure 15

Comparison of average revenue of the primary user with different risk attitudes by using Algorithm 1. In 500 independent channel realizations).

In Figure 15, risk-averse means that β₁=−1,β₂=−1,β₃=−1,β₄=−1, risk-neutral means that β₁=0,β₂=0,β₃=0,β₄=0, risk-seeking means that β₁=1,β₂=1,β₃=1,β₄=1, and the term ‘arbitrary’ means that β₁=0,β₂=−1,β₃=0,β₄=1. It is found that if the secondary users are all risk-averse, then the revenue of the primary users will be the lowest. If the secondary users are all risk-neutral or risk-seeking, then the primary user will receive higher revenue; the secondary users of ‘arbitrary types’ will result in modest revenue for the primary user.

6 Conclusions

In this paper, we have established a game theoretic framework for auction-driven dynamical spectrum access in cognitive radio networks with heterogeneous secondary users. We have studied the Nash equilibrium of the defined game and have proved that a unique pure-strategy Nash equilibrium exists in some particular situations. Due to the dynamic environment in cognitive radio networks, neither the primary user nor the secondary users have full information of the system. Thus, centralized algorithm is impractical. To attain an equilibrium solution with only incomplete information in a distributed setting, we have proposed a distributed learning automata algorithm. We have proved that our proposed algorithm can converge to a Nash equilibrium of our defined game with proper stepsize parameters. The simulation results have shown that our proposed algorithm can enable the auctioneer to obtain much higher revenue compared to the use of a fixed auction mechanism. At the same time, the proposed algorithm can significantly improve both the average sum capacities and the average sum utilities of secondary users.

Endnote

^a Ordinarily symmetric games are those that are symmetric with respect to the ordinal structure of the payoffs.