Advertisement

Mobile Networks and Applications

, Volume 16, Issue 2, pp 217–230 | Cite as

Distributed Adaptive Algorithms for Optimal Opportunistic Medium Access

  • Yahya Al-Harthi
  • Sem BorstEmail author
  • Phil Whiting
Open Access
Article

Abstract

We examine threshold-based transmission strategies for distributed opportunistic medium access in a scenario with fairly general probabilistic interference conditions. Specifically, collisions between concurrent transmissions are governed by arbitrary probabilities, allowing for a form of channel capture and covering binary interference constraints as an important special case. We address the problem of setting the threshold values so as to optimize the aggregate throughput utility of the various users, and particularly focus on a weighted logarithmic throughput utility function (Proportional Fairness). We provide an adaptive algorithm for finding the optimal threshold values in a distributed fashion, and rigorously establish the convergence of the proposed algorithm under mild statistical assumptions. Moreover, we discuss how the algorithm may be adapted to achieve packet-level stability with only limited exchange of queue length information among the various users. We also conduct extensive numerical experiments to corroborate the theoretical convergence results.

Keywords

distributed algorithm opportunistic scheduling proportional fairness queue stability random access  stochastic convergence threshold strategy throughput utility 

1 Introduction

Channel conditions in wireless networks exhibit huge variations across space and time, giving rise to vast random fluctuations in the feasible transmission rates. Channel-aware or opportunistic scheduling strategies provide an effective mechanism for improving throughput performance by exploiting such rate variations, and have attracted immense attention over the last decade.

Even though the seminal paper by Knopp and Humblet [15] considered opportunistic scheduling in an uplink scenario, the bulk of the work has focused on scheduled channel access and centralized control as is typical of downlink type situations, see for instance [3, 4, 6, 16, 22, 23, 30]. Extensions to joint scheduling and congestion control as well as routing in network settings have also been thoroughly investigated [5, 7, 21, 27, 28].

The problem of opportunistic scheduling in a distributed setting in conjunction with random access mechanisms such as Aloha has received relatively less attention. Adireddy and Tong [1, 2] and Venkitasubramaniam et al. [31] present strategies where the probability of transmission is a function of the Signal-to-Noise Ratio of a user. They study the problem of finding the transmission probabilities so as to maximize the aggregate throughput, characterize the maximum achievable throughput, and evaluate the throughput gains in various scenarios. Qin and Berry [24, 25] develop channel-aware Aloha type strategies as well as so-called splitting algorithms for exploiting multi-user diversity and improving the throughput performance in a distributed setting. Ghurumuruhan et al. [8, 9] consider the extension of channel-aware Aloha type strategies to multi-channel scenarios, and evaluate the throughput gains from the additional degree of channel diversity. Ghurumuruhan et al. [10] examine the impact of imperfect channel state information on the throughput performance of channel-aware Aloha strategies in multi-carrier systems. The stability region of multi-carrier systems with channel-aware Aloha type strategies is analyzed in Ghurumuruhan et al. [11]. Yu and Giannakis [35, 36] establish that threshold-based transmission strategies maximize the aggregate throughput in case of homogeneous users and the sum of the log throughputs in case of heterogeneous users.

The above papers offer interesting results for aggregate throughput measures in symmetric scenarios, but do not specifically address the issue as to how to set the various parameters, such as transmission threshold values, so as to optimize a given performance or throughput utility criterion in heterogeneous cases. The latter problem is explicitly addressed in a separate strand of papers which propose distributed algorithms for optimizing transmission probabilities for various given throughput objectives in the context of Aloha systems but without channel variations.

Kar et al. [13, 14] and Wang and Kar [33] were the first to present distributed random-access algorithms for achieving Proportional Fair and Max-Min fair rate allocations, respectively, in Aloha systems with single-hop flows. Gupta and Stolyar [12] develop distributed algorithms for realizing a weighted Proportional Fair throughput allocation subject to minimum throughput requirements in a single-hop Aloha networks with a more general interference structure. Stolyar [29] presents two distributed random-access algorithms based on queue lengths for similar single-hop Aloha models with exogenous packet arrivals, and shows that these guarantee stability as long as the arrival rate vector lies within the saturation rate region. Wang and Kar [34] present both primal-based and dual-based algorithms for achieving a Proportional Fair rate allocation in Aloha networks with multi-hop sessions. Liu and Stolyar [17] propose different random-access algorithms based on queue lengths and back-pressure mechanisms for achieving a weighted Proportional Fair allocation and guaranteeing stability in multi-hop Aloha networks.

In the present paper we examine threshold strategies for distributed channel-aware medium access as in [1, 2, 31, 36], where the users are subject to fairly general probabilistic interference conditions. Specifically, collisions between concurrent transmissions are governed by arbitrary probabilities, allowing for a form of channel capture and covering binary interference constraints as an important special case. We address the problem of setting the threshold values so as to optimize the aggregate throughput utility of the various users, and particularly focus on a weighted logarithmic throughput utility function (Proportional Fairness). We provide an adaptive algorithm for finding the optimal threshold values in a distributed fashion, and rigorously establish the convergence of the proposed algorithm under mild statistical assumptions. Moreover, we discuss how the algorithm may be adapted to achieve packet-level stability with only limited exchange of queue length information among the various users. We also perform extensive numerical experiments to support the theoretical convergence results.

The remainder of the paper is organized as follows. In Section 2 we present a model description and discuss a few important preliminaries. We analyze the throughput optimization problem in Section 3. In Section 4 we provide an adaptive algorithm for finding the optimal threshold values in a distributed fashion, and prove its convergence. Various packet-level stability issues are discussed Section 5. In Section 6 we present numerical results to further investigate the convergence properties of the proposed adaptive algorithm.

2 Model description

We consider a wireless multi-access channel shared by N users, operating in a time-slotted fashion. The feasible transmission rates of the various users vary over time as a result of fading. Denote by R i (t) the feasible transmission rate of user i in time slot t (measured for example in bits per second, bits per slot, or packets per slot). The feasible transmission rates are assumed to be independent among the various users and form a stationary sequence, i.e., R i (1), R i (2), ... have the same distribution as some generic random variable R i , but may not be independent across time. We further suppose that the distribution function \(F_i(r) = {\mathbb P}\{R_i < r\}\) is continuous with positive density f i (r) = dF i (r) / d r > 0. The users decide to access the channel in a distributed fashion, based on a threshold strategy. Specifically, there exists some threshold value γ i such that user i transmits in time slot t if R i (t) > γ i , regardless of the decisions of other users. Thus the probability that user i decides to transmit is τ i  = 1 − F i (γ i ), which we will also refer to as the activity factor of user i. When a user j ≠ i transmits in the same slot as user i, this causes the transmission of user i to fail with probability p ij , independent of anything else. The set S i  = {j: p ij  > 0} may be interpreted as the set of neighbors of user i in the interference ‘graph’, while the probabilities q ij  = 1 − p ij represent a form of channel capture allowing for arbitrary probabilistic interference constraints. Thus the transmission of user i in time slot t is successful if \({\prod_{j = 1}^{N}} {{\rm I}_{\{{R_{\!j}(t) \leq \gamma_{\!j} \mbox{ or } E_{ij}(t) = 0}\}}} = 1\), with E ij (t) representing a 0–1 random variable which equals 0 with probability q ij , independent of anything else, so that the probability that a given transmission of user i is successful, is \({\prod_{j = 1}^{N}} (q_{ij} + (1 - q_{ij}) F_{\!j}(\gamma_{\!j})) = (1 - p_{ij} (1 - F_{\!j}(\gamma_{\!j})))\), with the convention that p ii  = 0 for all i = 1, ..., N. The above-described threshold strategy yields a Pareto-optimal throughput vector within the class of ‘distributed’ strategies. In order to see that, consider a ‘distributed’ strategy where user i transmits a certain fraction τ i of the time. Given that user i transmits a fraction τ i of the time, independent of the activity of the other users (because of the ‘distributed’ nature), it makes stochastically no difference to the other users when user i is active. For user i itself, it is best to be active when its feasible transmission rate exceeds some threshold value γ i where τ i  = 1 − F i (γ i ). Thus, for any given ‘distributed’ strategy, one can construct a threshold-based strategy that provides no worse throughputs to each of the users.

For notational convenience, define
$$ U_i(\gamma_i) = {\mathbb E}\big\{R_i {{\rm I}_{\{{R_i > \gamma_i}\}}}\big\} = \int_{r = \gamma_i}^{\infty} r {\rm d}F_i(r), $$
and
$$ C_i(\gamma_i) = {\mathbb E}\big\{R_i | R_i > \gamma_i\big\} = \frac{U_i(\gamma_i)}{1 - F_i(\gamma_i)}. $$
The actual throughput or ‘goodput’ of user i in time slot t may be expressed as
$$ R_i(t) {{\rm I}_{\{{R_i(t) > \gamma_i}\}}} {\prod\limits_{j = 1}^{N}} {{\rm I}_{\{{R_{\!j}(t) \leq \gamma_{\!j} \mbox{ or } E_{ij}(t) = 0}\}}}, $$
and the expected throughput of user i is
$$\begin{array}{rll} T_i(\gamma) &=& U_i(\gamma_i) {\prod_{j = 1}^{N}} \big(q_{ij} + (1 - q_{ij}) F_{\!j}(\gamma_{\!j})\big) \\ &=& C_i(\gamma_i) (1 - F_i(\gamma_i)) {\prod_{j = 1}^{N}} \big(1 - p_{ij} (1 - F_{\!j}(\gamma_{\!j}))\big), \end{array}$$
with γ = (γ 1, ..., γ N ) representing the vector of threshold values.

3 Throughput optimization

As mentioned in the previous section, the above-described family of threshold strategies yield Pareto-optimal throughput vectors within the class of ‘distributed’ strategies. In this section we consider the problem of throughput optimization, i.e., finding a vector of threshold values which maximizes the aggregate throughput utility. We will focus on logarithmic throughput utility functions, which corresponds to weighted Proportional Fairness.

Denote by w i  > 0 the weight of user i in the throughput optimization. Then the aggregate throughput utility function may be expressed as
$$\begin{array}{rll} G(\gamma) &=& {\sum\limits_{i = 1}^{N}} w_i \log(T_i(\gamma_i)) \\ &=& {\sum\limits_{i = 1}^{N}} w_i \log(U_i(\gamma_i) {\prod_{j = 1}^{N}} \big(1 - p_{ij} (1 - F_{\!j}(\gamma_{\!j}))\big) \\ &=& {\sum\limits_{i = 1}^{N}} \! w_i \left[\log(U_i(\gamma_i)) \!+\! {\sum_{j = 1}^{N}} \log(1 \!-\! p_{ij} (1 \!-\! F_{\!j}(\gamma_{\!j})))\right] \\ &=& {\sum\limits_{i = 1}^{N}} \left[w_i \log(U_i(\gamma_i)) \phantom{{\sum_{j = 1}^{N}}}\right. \\ &&{\kern18pt} \left. + {\sum\limits_{j = 1}^{N}} w_{\!j} \log\big(1 \!-\! p_{\!ji} (1 \!-\! F_i(\gamma_i))\big)\right]. \end{array}$$
Noting that \(\frac{dU_i(\gamma_i)}{d\gamma_i} = - \gamma_i f_i(\gamma_i)\), we obtain
$$\begin{array}{rll} \frac{\partial G(\gamma)}{\partial \gamma_i} \!&=&\! \frac{w_i}{U_i(\gamma_i)} \frac{dU_i(\gamma_i)}{d\gamma_i} \!+\! {\sum_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 \!-\! p_{\!ji} (1 - F_i(\gamma_i))} \frac{dF_i(\gamma_i)}{d\gamma_i} \\ \!&=&\! - \Bigg[\frac{w_i \gamma_i}{U_i(\gamma_i)} \!-\! {\sum_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 - p_{\!ji} (1 - F_i(\gamma_i))}\Bigg] f_i(\gamma_i). \end{array}$$
Thus, the optimality condition becomes:
$$ \frac{w_i \gamma_i}{U_i(\gamma_i)} = {\sum_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 - p_{\!ji} (1 - F_i(\gamma_i))}. \label{opco1} $$
(1)
Note that the latter equation has a unique solution \(\gamma_i^*\), since the left-hand side is strictly increasing in γ i being equal to 0 for γ i  = 0 and approaching ∞ for γ i → ∞, while the right-hand side is nonnegative decreasing in γ i . In particular, when p ji  = 0 for all j ≠ i, the unique solution is \(\gamma_i^* = 0\). Indeed, it is optimal for user i to always transmit if it cannot cause any collisions to other users j ≠ i. A further important and remarkable property is that the optimal threshold value γ i only depends on the weights w j and transmission failure probabilities p ji of the other users, and not on their transmission rate distributions. Inspection shows that the optimal threshold value γ i is increasing (implying that the transmission strategy of user i becomes less aggressive) in the weights w j and transmission failure probabilities p ji of the other users.
In case p ji is either 0 or 1 for all j ≠ i, the optimality condition in Eq. 1 reduces to
$$ \frac{w_i \gamma_i}{U_i(\gamma_i)} = \frac{w_{- i}}{F_i(\gamma_i)}, $$
with \(w_{- i} := {\sum_{j = 1}^{N}} w_{\!j} p_{\!ji}\), or equivalently
$$ w_i \gamma_i F_i(\gamma_i) = w_{- i} U_i(\gamma_i). \label{opco2} $$
(2)
Now suppose that the transmission rate of user i has an exponential distribution with mean r i , i.e., \(F_i(r) = 1 - e^{- r / r_i}\). This assumption is roughly valid when the user has a Rayleigh fading channel and the feasible rate is approximately linear in the SNR (signal-to-noise ratio). The latter approximation is reasonably accurate when the SNR is not too high. In that case, it is easily verified that \(U_i(r) = (r + r_i) e^{- r / r_i}\), and the optimality condition 2 takes the form:
$$ w_i \gamma_i \left(1 - e^{- \gamma_i / r_i}\right) = w_{-i} \big(\gamma_i + r_i\big) e^{- \gamma_i / r_i}, $$
or equivalently
$$ \gamma_i / r_i = \big(w_{-i} + (w_i + w_{-i}) \gamma_i / r_i\big) e^{- \gamma_i / r_i}, $$
which means that \(\gamma_i^* = r_i s_i^*\), with \(s_i^*\) the solution of the equation
$$ s = \big(w_{-i} + (w_i + w_{-i}) s\big) e^{- s}, \label{eqss1} $$
(3)
and the corresponding activity factor is \(\tau_i = e^{- s_i^*}\), independent of the mean transmission rate r i . In particular, for symmetric weights, the optimal threshold values are proportional to the mean feasible rates of the various users, where the factor of proportionality is the solution of the equation
$$ s = \big(N - 1 + N s\big) e^{- s}. \label{eqss2} $$
(4)
Substituting U i (γ i ) = C i (γ i ) (1 − F i (γ i )) in Eq 2, we obtain w i γ i F i (γ i ) = w  − i C i (γ i ) (1 − F i (γ i )), or equivalently, w i γ i  =  (w i γ i  + w  − i C i (γ i )) (1  −  F i (γ i )), which gives \(1 - F_i(\gamma_i) = \frac{w_i \gamma_i}{w_i \gamma_i + w_{- i} C_i(\gamma_i)}\). Noting that C i (γ i ) ≥ γ i , we deduce in case p ji  = 1 for all j ≠ i,
$$ 1 - F_i(\gamma_i) \leq \frac{w_i}{{\sum_{j = 1}^{N}} w_{\!j}}, $$
and
$$ {\sum\limits_{i = 1}^{N}} \big(1 - F_i(\gamma_i)\big) \leq 1. $$
Hence, the collective activity factor of all the users is at most unity, and if the user’s weights are all equal, then the activity factor of each individual user is at most 1 / N. Recall that in the standard Aloha system without channel variations, the sum of the transmission probabilities of all the users is exactly equal to unity for every Pareto-optimal throughput vector [19]. Thus, the use of opportunistic medium access in the presence of channel variations reduces the optimal collective activity factor, essentially because the relative loss caused by collisions is larger.
In a symmetric scenario, it may be deduced that the optimal collective activity factor is
$$ \frac{N \gamma}{\gamma + M C(\gamma)}, $$
with \(M := {\sum_{j = 1}^{N}} p_{\!ji} \leq N - 1\). In particular, the optimal collective activity factor may either be smaller or larger than unity, depending on whether (N  −  1) γ is smaller or larger than M C(γ), or equivalently whether M / (N  −  1) is smaller or larger than γ/ C(γ). Note that M / (N  −  1) provides a measure for the degree of interference among the users, while C(γ) / γ offers a proxy for the amount of channel variation.
As an alternative approach, we may work with the activity factors τ i  = 1 − F i (γ i ) as the independent variables instead of the threshold values, and write, with minor abuse of notation, \(U_i(\tau_i) = {\mathbb E}\{R_i {{\rm I}_{\{{R_i > F_i^{- 1}(1 - \tau_i)}\}}}\}\). In that case, the expression for the expected throughput of user i is \(T_i(\tau) = U_i(\tau_i) {\prod_{j = 1}^{N}} (1 - p_{ij} \tau_{\!j})\) with τ = (τ 1, ..., τ N ). The expression for the aggregate throughput utility function is:
$$\begin{array}{rll} G(\gamma) &=& {\sum\limits_{i = 1}^{N}} w_i \log(T_i(\gamma_i)) \\ &=& {\sum\limits_{i = 1}^{N}} \Bigg[w_i \log(U_i(\tau_i)) + {\sum\limits_{j = 1}^{N}} w_{\!j} \log(1 - p_{\!ji} \tau_i)\Bigg]. \end{array}$$
Noting that \(\frac{dU_i(\tau_i)}{d\tau_i} = F^{- 1}(1 - \tau_i)\), we obtain
$$\begin{array}{rll} \frac{\partial G(\tau)}{\partial \tau_i} &=& \frac{w_i}{U_i(\tau_i)} \frac{dU_i(\tau_i)}{d\tau_i} - {\sum\limits_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 - p_{\!ji} \tau_i} \\ &=& \frac{w_i F_i^{- 1}(1 - \tau_i)}{U_i(\tau_i)} - {\sum\limits_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 - p_{\!ji} \tau_i}. \end{array}$$
Thus the optimality condition becomes
$$ \frac{w_i F_i^{- 1}(1 - \tau_i)}{U_i(\tau_i)} = {\sum\limits_{j = 1}^{N}} \frac{w_{\!j} p_{\!ji}}{1 - p_{\!ji} \tau_i}, $$
which corresponds to Eq. 1 as F i (γ i ) = 1 − τ i .

Remark 3.1

When we work with the activity factors instead of the threshold values as the independent variables, we can in fact allow for rate distribution functions that are not continuous but have probability mass in discrete points. Specifically, we can define \(U_i(\tau_i) \!=\! \int_{r = 0}^{\gamma_i(\tau_i)}\! r {\rm d}F_i(r) \!+\! \gamma_i(\tau_i) (\tau_i \!-\! F_i(\gamma_i^-(\tau_i))\), with γ i (τ i )  = sup{γ: F i (γ) ≤ τ i }. In particular, we can allow for a given user i to have a constant transmission rate r i , in which case \(F_i^{- 1}(s) \equiv r_i\) and U i (τ i ) ≡ r i τ i , so that the optimality condition reduces to w i (1 − τ i ) = w  − i τ i in case p ji is either 0 or 1 for all i ≠ j. In case w i  = 1 for all i = 1, ..., N, we obtain τ i  = 1 / N, which is consistent with the fact that in a standard Aloha system with constant (unit) transmission rates the sum of the transmission probabilities of all users is equal to unity for every Pareto-optimal throughput vector as mentioned earlier [19].

Remark 3.2

As observed in the Introduction, the present paper provides a bridge between opportunistic medium access strategies in the presence of channel variations and distributed throughput utility optimization in a standard Aloha system without channel variations. Indeed, the expression for the throughput of user i in terms of the activity factors τ j has a similar structure as in a standard Aloha system (with the term U i (τ i ) rather than just τ i accounting for the gains from opportunistic medium access), which is again related to the observation in the previous remark.

4 Algorithm specification

As observed before, the optimality condition 1 only involves the rate distribution function F i (·) of user i, and the weights w j and transmission failure probabilities p ji of the other users, but not their rate distribution functions F j (·). The optimal threshold values \(\gamma_i^*\) can therefore be found in a distributed fashion, as long as each user i knows the weights w j and transmission failure probabilities p ji of the other users. Particularly, in case p ji is either 0 or 1 for all j ≠ i, user i only needs to know the aggregate weight \(w_{- i} = {\sum_{j = 1}^{N}} w_{\!j} p_{\!ji}\) of the other users it interferes with. When the rate distribution function F i (·) and hence the function U i (·) are known, Eq. 1 can then in principle be solved using numerical means. In practice, however, the function F i (·) will not be known exactly and may exhibit non-stationary characteristics, which means that use of an adaptive algorithm based on actual rate observations is preferable.

We now proceed to specify such an algorithm, where we focus on a window-based version, but the basic concept easily extends to similar non-window-based incarnations. For convenience, we first treat the case where p ji is either 1 or 0 for all j ≠ i, so that the optimality condition 2 applies. Below we will discuss the case of arbitrary values of p ij .

We assume that user i operates in cycles of possibly variable lengths, where the k-th cycle consists of L i (k) ≥ 1 consecutive time slots. Let γ i (k) be the threshold value of user i at the beginning of the k-th cycle. In each slot of the k-th cycle, user i transmits if and only if its instantaneous rate exceeds γ i (k). Denote by R i (k, l) the feasible transmission rate of user i in the l-th time slot of the k-th cycle. Let the 0–1 variable \(X_i(k, l) = \text{I}_{\{R_i(k, l) > \gamma_i(k)\}}\) indicate whether user i transmits in the l-th time slot of the k-th cycle or not. At the end of the k-th cycle, we compute the fraction \(\hat{F}_i(k) = \frac{1}{L_i(k)} \sum_{l = 1}^{L_i(k)} (1 - X_i(k, l))\) as empirical estimate for F i (γ i (k)) and \(\hat{U}_i(k) = \frac{1}{L_i(k)} \sum_{l = 1}^{L_i(k)} X_i(k, l) R_i(k, l)\) as empirical estimate for U i (γ i (k)).

In view of the optimality condition 2, we next calculate the difference \(D_i(k) = w_i \gamma_i(k) \hat{F}_i(k) - w_{- i} \hat{U}_i(k)\). In case D i (k) < 0, the current threshold value γ i (k) is likely to be smaller than the optimal threshold value \(\gamma_i^*\), whereas in the opposite case, it is likely to be larger. Hence, in the former case it makes sense to increase the threshold, whereas in the latter case it is sensible to decrease the threshold. We will consider two specific options, (A) multiplicative updates with a constant factor and (B) additive updates with a variable step size, but alternative combinations are possible too. In option (A), we set the threshold value for the next cycle to γ i (k + 1) = γ i (k) (1 + ε) or γ i (k + 1) = γ i (k) (1 − ε), with ε > 0 a suitably small coefficient, depending on whether D i (k) is negative or positive, respectively. In option (B), we set the threshold value for the next cycle to γ i (k + 1) = γ i (k) − δ i (k) D i (k) with δ i (k) > 0 a sequence of coefficients such that \(\sum_{k = 0}^{\infty} \delta_i(k) = \infty\), \(\sum_{k = 0}^{\infty} \delta_i(k)^2 < \infty\).

In view of the drift condition, it is intuitively plausible that in both versions the threshold will approach the optimal value \(\gamma_i^*\) in some sense. In option (A), assuming L i (k) ≡ L i for all k = 1, 2, ..., the threshold value will converge to a random variable which will continue to toggle around \(\gamma_i^*\), with the magnitude of the oscillation governed by the value of ε. In option (B), the threshold value will converge to \(\gamma_i^*\) almost surely, as we will rigorously establish in the theorem below. (For mathematical convenience, we will make the common assumption that the feasible transmission rates are independent over time, but the proof arguments can be extended to scenarios where the rates show temporal correlations. A detailed convergence proof for option (A) would require a complex and lengthy mathematical treatment, and go beyond the scope of the present paper.) From a practical point of view, however, the downside of (B) is the sluggish response to changes in the underlying statistics, whereas option (A) is less accurate but more responsive. In Section 6 we will conduct numerical experiments to examine the performance of both versions (A) and (B).

Theorem 4.1

(Convergence of thresholds)

Assume that the feasible transmission rates R i (1),R i (2), ... are i.i.d. copies of a continuous random variable R i with\({{\mathbb E}\left\{{R_i^2}\right\}} < \infty\). Let δ i (k) > 0 be a sequence of coefficients such that \(\sum_{k = 0}^{\infty} \delta_i(k) = \infty\), \(\sum_{k = 0}^{\infty} \delta_i(k)^2 < \infty\). Let γ i (k) be the value of the threshold of user i after k updates according to \(\gamma_i(k + 1) = \left[\gamma_i(k) - \delta_i(k) D_i(k)\right]_0^{\gamma_i^{\max}}\), with
$$ [x]_a^b = \left\{\begin{array}{ll} a \mbox{ if } x < a \\[1.5pt] x \mbox{ if } a \leq x \leq b \\[1.5pt] b \mbox{ if } x > b \end{array} \right., $$
\(\gamma_i^{\max} > \gamma_i^*\) an arbitrary constant, and D i (k) as defined above. Let \(\gamma_i^*\)be the optimal threshold value of user i, i.e., the unique solution of Eq.2, and assume that
$$ \mu_i = \inf_{\gamma_i \neq \gamma_i^*} \frac{w_i \gamma_i F_i(\gamma_i) - w_{- i} U_i(\gamma_i)} {\gamma_i - \gamma_i^*} > 0. $$
Then the random sequence γ i (k) converges to\(\gamma_i^*\)almost surely as k → ∞.

The proof of Theorem 4.1 relies on a powerful convergence result for nonnegative ‘almost’ supermartingales which is stated in the next theorem.

Theorem 4.2

(Convergence of nonnegative ‘almost’ supermartingales)

Let the random sequence M k  ≥ 0, k = 1, 2, ⋯ be an ‘almost’ supermartingale i.e., the sequence satisfies the following three conditions:
  1. (i)

    \(\sup_k {{\mathbb E}\left\{{M_k}\right\}} < \infty\) ;

     
  2. (ii)

    M k is adapted to a filtration (increasing sequence of σ-algebras) \({\cal F}_1 \!\subseteq\! \cdots \!\subseteq\! {\cal F}_{k-1} \!\subseteq\! {\cal F}_k \!\subseteq\! \ldots\), where the k-th σ-algebra, \({\cal F}_k\) is that induced by M1,M2, ⋯ , M k ;

     
  3. (iii)

    \({{\mathbb E}\left\{{M_{k+1} | {\cal F}_k}\right\}} \leq M_k + \epsilon_{k+1}\), for some sequence of coefficients ε k  ≥ 0 with \(\sum_{k = 1}^{\infty} \epsilon_k < \infty\).

     

Then limk→ ∞ M k  = M ∞  exists and is finite with probability 1, where M ∞  is measurable with respect to \({\cal F}_\infty = \cup_{k = 1}^{\infty} {\cal F}_k\).

The proof of Theorem 4.2 follows by using the definition of an ‘almost’ supermartingale and following the steps leading to Theorem 11.5 in [32], see also Chapter 10, and [20]

We are now in a position to prove Theorem 4.1.

Proof of Theorem 4.1

Define \(Z_k := \gamma_i(k) - \gamma_i^*\), where we drop the user index i for conciseness. We will show that the sequence \(Z_k^2\) converges to 0 almost surely, which implies the almost-sure convergence of the thresholds γ i (k) to \(\gamma_i^*\) as k → ∞.

The proof consists of two parts. We will first show that the sequence \(Z_k^2\) converges to a finite limit almost surely by verifying that it is an almost supermartingale and then invoking Theorem 4.2. Second, we will prove that the latter limit must be 0.

We first verify that the sequence \(Z_k^2\) satisfies conditions (i)–(iii).
  1. (i)

    This follows from (iii).

     
  2. (ii)

    Adapted with respect to \({\cal F}_k\) means that the Z k events are contained within \({\cal F}_k\). This follows by construction of the sequence γ i (k).

     
  3. (iii)
    We may write γ i (k + 1) = γ i (k) − δ i (k) ζ i (k) D i (k), with \(\zeta_i(k) = \frac{\gamma_i(k) - \gamma_i(k+1)}{\delta_i(k) D_i(k)} \in [0, 1]\), and thus
    $$\begin{array}{rll} Z_{k+1} &=&\gamma_i(k + 1) - \gamma_i^* \\ &=& \gamma_i(k) - \delta_i(k) \zeta_i(k) \hat{D}i(k) - \gamma_i^* \\ &=& Z_k -\delta_i(k) \zeta_i(k) D_i(k), \end{array}$$
    yielding
    $$\begin{array}{rll} {{\mathbb E}\left\{{Z_{k+1}^2 | {\cal F}_k}\right\}} &=& Z_k^2 - 2 \delta_i(k) \\ &&\times\,{{\mathbb E}\left\{{(\gamma_i(k) - \gamma_i^*) \zeta_i(k) D_i(k)}\right\}} \\ &&+\, \delta_i(k)^2 {{\mathbb E}\left\{{\zeta_i(k)^2 D_i(k)^2}\right\}}. \end{array}$$
     

If \(\gamma_i(k) \leq \gamma_i^*\), then \({{\mathbb E}\left\{{D_i(k)}\right\}} \leq 0\), and if \(\gamma_i(k) \geq \gamma_i^*\), then \({{\mathbb E}\left\{{D_i(k)}\right\}} \geq 0\), so that \({{\mathbb E}\left\{{(\gamma_i(k) - \gamma_i^*) \zeta_i(k) D_i(k)}\right\}} \geq 0\). Hence \({{\mathbb E}\left\{{Z_{k+1}^{2} | {\cal F}_k}\right\}} \leq Z_{k}^{2} + \epsilon_{k+1}\), with \(\epsilon_{k+1} = \delta_i(k)^2 \). Now observe that \(D_i(k) \in [- \frac{w_{-i}}{L_i(k)}\sum_{l = 1}^{L_i(k)} R_i(k), w_i \gamma_i(k)]\), which gives \({{\mathbb E}\left\{{\zeta_i(k)^2 D_i(k)^2}\right\}} \leq {{\mathbb E}\left\{{D_i(k)^2}\right\}} \leq \max\{ w_{-i}^2 {{\mathbb E}\left\{{R_i^2}\right\}}, w_i^2 (\gamma_i^{\max})^2\}\). The fact that \(\sum_{k = 0}^{\infty} \delta_i(k)^2 < \infty\) then implies that \(\sum_{k = 0}^{\infty} \epsilon_k < \infty\).

Applying Theorem 4.2, we conclude that the sequence \(Z_k^2\) converges to a finite limit \(Z_\infty^2\) almost surely.

In order to complete the proof, it remains to be shown that \(Z_\infty^2 = 0\).

First observe that
$$\begin{array}{rll} \vert Z_{k+1} \vert &=& \vert \gamma_i(k+1) - \gamma_i^* \vert \\ &=& \vert \gamma_i(k) - \gamma_i^* - \delta_i(k) \zeta_i(k) D_i(k) \vert \\ &\leq& \vert \gamma_i(k) - \gamma_i^* - \delta_i(k) D_i(k) \vert. \end{array}$$
It follows that
$$\begin{array}{rll} {{\mathbb E}\left\{{Z_{k+1}^2 | {\cal F}_k}\right\}} &\leq& Z_k^2 - 2 \delta_i(k) {{\mathbb E}\left\{{Z_k D_i(k)}\right\}} \\ &&+\, \delta_i(k)^2 {{\mathbb E}\left\{{D_i(k)^2}\right\}}, \end{array}$$
which could in fact also have been deduced from the arguments used in checking condition (iii) above.
Taking expectations in the above inequality, telescoping over all k, and recalling that \(\sum_{k = 0}^{\infty} \delta_i(k)^2 < \infty\) and \(\sup_k {{\mathbb E}\left\{{D_i(k)^2}\right\}} < \infty\), we obtain
$$\begin{array}{rll} 2 \sum\limits_{k=0}^{\infty} \delta_i(k) {{\mathbb E}\left\{{Z_k D_i(k)}\right\}} &\leq& {{\mathbb E}\left\{{Z_0^2}\right\}} - {{\mathbb E}\left\{{Z_\infty^2}\right\}} \\&&+ \sum\limits_{k=0}^\infty \delta_i(k)^2 {{\mathbb E}\left\{{D_i(k)^2}\right\}} < \infty. \end{array}$$

Also, \({{\mathbb E}\left\{{Z_k D_i(k)}\right\}} \geq \mu_i {{\mathbb E}\left\{{Z_k^2}\right\}}\).

Since \(\sum_{k = 0}^{\infty} \delta_i(k) = \infty\), it follows that
$$ \liminf_{k \to \infty} {{\mathbb E}\left\{{Z_k^2}\right\}} = 0. $$
Invoking Fatou’s lemma, we deduce
$$ {{\mathbb E}\left\{{Z_\infty^2}\right\}} = {{\mathbb E}\left\{{\liminf_{k \to \infty} Z_k^2}\right\}} \leq \liminf_{k \to \infty} {{\mathbb E}\left\{{Z_k^2}\right\}} = 0, $$
which implies that \(Z_\infty^2 = 0\) almost surely. □

We now briefly treat the case of probabilistic capture, where we suppose that all capture probabilities satisfy 0 < p ij  < 1, j ≠ i, and are known exactly. Assume further that L rate samples are used before each update.

We will concentrate on user i and suppress the i index as well as the cycle index k. Let X l , l = 1, ⋯ , L, be the indicator random variables as before, so that \({{\mathbb E}\left\{{X_l}\right\}} \!=\! 1 \!-\! F(\gamma)\). We now show that similar arguments to the above will lead to convergence to a point arbitrarily close (of O(1/L) to the root of Eq. 1).

Define
$$ A_L(\gamma, X) := \frac{\frac{1}{L} \sum_{l = 1}^{L} R_l X_l}{\gamma}, $$
then \({{\mathbb E}\left\{{A_L}\right\}}\) is decreasing in γ. Also, define
$$ B_L(\gamma, X) := \left(\sum\limits_{j=1}^{N} \frac{w_{\!j}p_{\!j}} {1-\frac{p_{\!j}}{L} \sum_{l = 1}^{L} X_l}\right)^{-1}, $$
so that \({{\mathbb E}\left\{{B_L}\right\}}\) is increasing in γ and the two expectations have a root 0 < γ min < γ L  < γ max, for sufficiently large L. Hence, if we set D L : = B L  − A L we obtain an ‘almost’ supermartingale as before.
Since at the root
$$\begin{array}{lll}{{\mathbb E}\left\{{A_L}\right\}} &=& \frac{U(\gamma_L)}{\gamma_L} \\ {{\mathbb E}\left\{{B_L}\right\}} &=& \left(\sum_{j=1}^{N} \frac{w_{\!j}p_{\!j}} {1-p_{\!j}(1-F(\gamma_L)}\right)^{-1} + \varepsilon_L \end{array}$$
where ε L →0 as L → ∞, the point of convergence can be chosen arbitrarily close to γ *.

5 Packet-level stability

In the previous section we considered the problem of throughput optimization, implicitly assuming the various users to have saturated queues, i.e., always have packets to transmit, and established the convergence of the proposed adaptive algorithm. In this section we turn our attention to a scenario with queue dynamics where users are not infinitely backlogged but have queues fed by packet arrivals, and address the stability of the adaptive transmission algorithm. Specifically, denote by A i (t) the number of packets arriving to the queue of the ith user in the tth time slot. We assume that A i (1), A i (2), ... are i.i.d. copies of some discrete random variable A i with mean \(\lambda_i = {\mathbb E}\{A_i\}\). We assume that a user only decides to transmit if its queue is non-empty and its feasible transmission rate exceeds the threshold value. Since the arrival processes are expressed in terms of packets per slot, it will be convenient to measure the feasible transmission rates in the same unit.

Let Q i (t) represent the queue length of the ith user at the start of the tth time slot. Define
$$ R_i'(t) = R_i(t) X_i(t) \prod\limits_{j \in S_i} (1 - X_{\!j}(t)), $$
as the number of packets served from the ith queue in the tth time slot, with
$$ X_i(t) = {{\rm I}_{\{{R_i(t) > \gamma_k, Q_i(t) + A_i(t) > 0}\}}} $$
indicating whether or not the ith user transmits in the tth time slot. The time evolution of the ith queue may then be described by the familiar recursion
$$ Q_i(t+1) = \big[Q_i(t) + A_i(t) - R_i'(t)\big]^+. $$
Observe that the queue size process Q(t) = (Q 1(t),..., Q N (t)) is a discrete-time Markov chain.

An important issue that arises, is (I) whether the queue process is ‘stable’ in some appropriate sense. More formally stated, for a given vector of threshold values γ = (γ 1, ..., γ N ), what is the stability region \(\Lambda(\gamma) \in {{\mathbb R}}_+^N\), i.e., the set of arrival rate vectors λ = (λ 1, ..., λ N ) for which the queue process is stable? A closely related question is (II) whether we can easily characterize the overall stability region, \(\Lambda = \cup_{\gamma \in {{\mathbb R}}_+^N} \Lambda(\gamma)\), i.e., the set of arrival rate vectors for which there exists a vector of threshold values such that the queues are stable. A further related question is (III) whether we can find threshold values that achieve stability, given that λ ∈ Λ, but without any explicit knowledge of the feasible transmission rate distributions and arrival parameters.

We will address each of the problems (I), (II) and (III) below.

5.1 Problem I

Problem I has been thoroughly studied in the slightly simpler but similar context of a standard slotted Aloha system (without channel variations), where the stability region is determined for a given vector of transmission probabilities, rather than for a given vector of threshold values. While an abundance of interesting results have been obtained in the literature, an explicit characterization of the stability region in that setting is essentially limited to the case of two users. The dependence between the queue lengths renders the case of an arbitrary number of users extremely complex, and the available results are limited to bounds and asymptotic approximations, or entail restrictive assumptions on the packet arrival processes.

In view of these considerations, we henceforth focus on the case of two interfering users, i.e., N = 2, and use a similar proof technique as developed for the standard slotted Aloha model. In particular, we adopt the notion of a ‘dominant system’ as in Luo and Ephremides [18], Rao and Ephremides [26] and consider a corresponding version of the system where user 1 transmits ‘dummy’ packets when its queue is empty. It is easily verified that the queues of both users are stochastically larger in that system, hence the term dominant system. In particular, stability of the queue process in the dominant system implies that of the queue process in the original system.

In the dominant system, the probability that a transmission of user 1 is successful, alternates between the values 1 and 1 − p 12 + p 12 F 2(γ 2), depending on whether the queue of user 2 is empty or not. In contrast, the probability that a transmission of user 2 is successful, is always 1 − p 21 + p 21 F 1(γ 1), since user 1 transmits dummy packets when its queue is empty, and hence the expected throughput of user 2 is T 2(γ 1, γ 2) = U 2(γ 2) (1 − p 21 + p 21 F 1(γ 1)). It follows that the queue of user 2 is stable if and only if
$$ \lambda_2 < U_2(\gamma_2) (1 - p_{21} + p_{21} F_1(\gamma_1)), \label{Eq:stable_11} $$
(5)
and whenever that inequality holds, the fraction of time that the queue is non-empty is
$$ \pi_2 = \frac{\lambda_2}{U_2(\gamma_2) \big(1 - p_{21} + p_{21} F_1(\gamma_1)\big)}. $$
Thus, the time-average throughput of user 1 is
$$\begin{array}{lll} &&{\kern-6pt} U_1(\gamma_1) \big[1 - \pi_2 + \pi_2 \big(1 - p_{12} + p_{12} F_2(\gamma_2)\big)\big] \\[4pt] &&{\kern6pt} = U_1(\gamma_1) \big[1 - \pi_2 p_{12} (1 - F_2(\gamma_2))\big] \\[4pt] &&{\kern6pt} = U_1(\gamma_1) \left[1 - \frac{\lambda_2 p_{12} (1 - F_2(\gamma_2))} {U_2(\gamma_2) (1 - p_{21} + p_{21} F_1(\gamma_1))}\right], \end{array}$$
which means that the queue of user 1 is stable if and only if
$$ \lambda_1 < U_1(\gamma_1) \left[1 - \frac{\lambda_2 p_{12} (1 - F_2(\gamma_2))} {U_2(\gamma_2) (1 - p_{21} + p_{21} F_1(\gamma_1))}\right]. \label{Eq:stable_12} $$
(6)
Because of the dominance, the above two inequalities imply that the queue process in the original system is stable.
Considering a dominant system with the roles of the two users reversed, we deduce that the queue process in the original system is also guaranteed to be stable when the following two inequalities are satisfied:
$$ \lambda_1 < U_1(\gamma_1) \big(1 - p_{12} + p_{12} F_2(\gamma_2)\big), \label{Eq:stable_21} $$
(7)
and
$$ \lambda_2 < U_2(\gamma_2) \left[1 - \frac{\lambda_1 p_{21} (1 - F_1(\gamma_1))} {U_1(\gamma_1) \big(1 - p_{12} + p_{12} F_2(\gamma_2)\big)}\right]. \label{Eq:stable_22} $$
(8)

We obtain that the stability of the queue length process is ensured when either inequalities 5 and 6 or inequalities 7 and 8 hold. Following a similar line of argument as in [18, 26], it may be concluded that the latter conditions are not only sufficient, but in fact also necessary for stability.

In conclusion, the stability region Λ(γ 1, γ 2) for a given pair of threshold values (γ 1, γ 2) is the set of all arrival rate vectors \((\lambda_1, \lambda_2) \in {{\mathbb R}}_+^2\) satisfying either inequalities 5 and 6 or inequalities 7 and 8.

5.2 Problem II

We now turn to the problem of characterizing the overall stability region \(\Lambda := \bigcup_{(\gamma_1, \gamma_2) \in {{\mathbb R}}_+^2} \Lambda(\gamma_1, \gamma_2)\), i.e., the set of arrival rate vectors for which there exists a pair of threshold values such that the queues are stable.

For any \((\gamma_1, \gamma_2) \in {{\mathbb R}}_+^2\), define \({\cal T}(\gamma_1, \gamma_2) := \{(x_1, x_2) \in_+^2: x_i \,\leq\, T_i(\gamma_1, \gamma_2)\} \,=\, [0, T_1(\gamma_1, \gamma_2)] {\kern.5pt} \times {\kern.5pt} [0, T_2(\gamma_1, \gamma_2)]\), and \({\cal T} := \bigcup_{(\gamma_1, \gamma_2) \in {{\mathbb R}}_+^2} {\cal T}(\gamma_1, \gamma_2)\). Note that \({\cal T}(\gamma_1, \gamma_2) \subseteq {\rm closure}(\Lambda(\gamma_1, \gamma_2))\).

Let g 1(w 1 / w 2) be the unique solution to the optimality equation w 1 γ 1 (1  −  p 21 + p 21 F 1(γ 1))  =  w 2 p 21 U 1(γ 1) as function of the ratio w 1 / w 2. Likewise, let g 2(w 2 / w 1) be the unique solution to the optimality equation w 2 γ 2 (1 − p 12 + p 12 F 2(γ 2)) = w 1 p 12 U 2(γ 2) as function of the ratio w 2 / w 1. For any r ∈ (0, ∞ ), define \({\cal U}(r) :=\{(x_1, x_2) \,\in\, {{\mathbb R}}_+^2: x_i \,\leq\, T_i(g_1(r), g_2(1/r))\} = [0, T_1(g_1(r),g_2(1/r))] \!\times\! [0, T_{\!2}(g_1(r), g_2(1/r))]\), and \({\cal U} \!:=\! \bigcup_{r \in (0, \infty)}\! U(r)\). It may then be shown that \({\rm closure}(\Lambda) = {\cal T} = {\cal U}\).
  1. (i)

    We first prove that \({\rm closure}(\Lambda) \subseteq {\cal T}\).

    Assume that (λ 1, λ 2) ∈ closure(Λ). Then there exists a vector \((\gamma_1, \gamma_2) \in {{\mathbb R}}_+^2\) such that (λ 1, λ 2) satisfies either inequalities 5 and 6 or inequalities 7 and 8. By symmetry, we may assume the former to be the case, without loss of generality. Now let γ 2′ be such that 1 − F 2(γ 2′) = π 2 (1 − F 2(γ 2)), with
    $$ \pi_2 = \frac{\lambda_2}{U_2(\gamma_2) (1 - p_{21} + p_{21} F_1(\gamma_1))} < 1, $$
    due to Eq. 5, so that γ 2 ≤ γ 2′, 1  −  p 21 + p 21 F 1(γ 1)  = λ 2 / (π 2 U 2(γ 2)) and
    $$\begin{array}{lll} &&{\kern-6pt}1 - p_{12} + p_{12} F_2\left(\gamma_2'\right)\\ &&{\kern6pt}= 1 - \pi_2 p_{12} (1 - F_2(\gamma_2)) \\&&{\kern6pt}= 1 - \frac{\lambda_2 p_{12} (1 - F_2(\gamma_2))} {U_2(\gamma_2) \big(1 - p_{21} + p_{21} F_1(\gamma_1)\big)}. \end{array}$$
    Thus,
    $$\begin{array}{rll} T_2\left(\gamma_1, \gamma_2'\right) &=& U_2\left(\gamma_2'\right) \big(1 - p_{21} + p_{21} F_1(\gamma_1)\big) \\ &=& \frac{\lambda_2 U_2\left(\gamma_2'\right)}{\pi_2 U_2(\gamma_2)} > \lambda_2 / \pi_2 > \lambda_2, \end{array}$$
    and
    $$\begin{array}{lll} &&{\kern-8pt} T_1\left(\gamma_1, \gamma_2'\right) \\&&{\kern3pt}= U_1(\gamma_1) \left(1 - p_{12} + p_{12} F_2\left(\gamma_2'\right)\right) \\ &&{\kern3pt}= U_1(\gamma_1) \bigg[\!1 \!-\! \frac{\lambda_2 p_{12} (1 \!-\! F_2(\gamma_2))} {U_2(\gamma_2) (1 \!-\! p_{21} \!+\! p_{21} F_1(\gamma_1))}\!\bigg] \!>\! \lambda_1, \end{array}$$
    because of Eq. 6.

    Hence, \((\lambda_1, \lambda_2) \in {\cal T}(\gamma_1, \gamma_2') \subseteq {\cal T}\).

     
  2. (ii)

    We now show that \({\cal T} \subseteq {\cal U}\).

    Assume that \((\lambda_1, \lambda_2) \in {\cal T}\). Then there exists a vector \((\gamma_1, \gamma_2) \in {{\mathbb R}}_+^2\) such that λ i  ≤ T i (γ 1, γ 2), i = 1, 2, and (T 1′, T 2′) ≥ (T 1(γ 1,γ 2), T 2(γ 1,γ 2)) implies that \((T_1', T_2')\notin {\cal T}\). Therefore \((\log(T_1(\gamma_1, \gamma_2)),\log(T_2(\gamma_1, \gamma_2))) \in \log({\cal T})\), with \(\log({\cal T})\! := \! \{(\log(T_1),\log(T_2)): (T_1, T_2) \in {\cal T}\}\), and (log(T 1′), log(T 2′)) ≥ (log(T 1(γ 1, γ 2)), log(T 2(γ 1, γ 2))) implies \((\log(T_1'),\log(T_2')) \notin \log({\cal T})\), i.e., (log(T 1(γ 1, γ 2)), log(T 2(γ 1,γ 2))) is Pareto-optimal and maximizes some linear function over \(\log({\cal T})\) since the set \(\log({\cal T})\) is convex as may be shown by a somewhat tedious but otherwise straightforward argument. In other words, there exists a vector \((w_1, w_2) \in {{\mathbb R}}_+^2\) such that
    $$\begin{array}{lll} &&{\kern-6pt} w_1 \log\big(T_1(\gamma_1, \gamma_2)\big) + w_2 \log\big(T_2(\gamma_1, \gamma_2)\big) \\ &&{\kern6pt}= \max_{(y_1, y_2) \in \log({\cal T})} w_1 y_1 + w_2 y_2. \end{array}$$
    It follows that γ 1 = g 1(r) and γ 2 = g 2(1 / r) with r = w 1 / w 2, i.e., \((\lambda_1, \lambda_2) \in {\cal U}(r) \subseteq {\cal U}\).
     
  3. (iii)

    Finally, we prove that \({\cal U} \subseteq {\rm closure}(\Lambda)\).

    Assume that \((\lambda_1, \lambda_2) \in {\cal U}\). Then there exists a number r ∈ (0, ∞ ) such that λ i  ≤ T i (g 1(r), g 2(1 / r)), i = 1, 2. Taking γ 1 = g 1(r) and γ 2 = g 2(1 / r), we obtain \((\lambda_1, \lambda_2) \!\in\! {\cal T}(\gamma_1, \gamma_2) \subseteq {\rm closure}(\Lambda(\gamma_1, \gamma_2)) \subseteq(\Lambda)\).

     

5.3 Problem III

Finally, we consider the problem of finding threshold values that achieve stability for an unknown vector of arrival rates λ (assuming λ ∈ Λ to ensure the existence of such threshold values). Surprisingly, the problem turns out to be simpler than the problem of determining the stability region for given threshold values, and can be tackled in far greater generality. Thus we now return to the scenario as studied in Sections 2 and 3 with an arbitrary number of N users and a general interference structure in terms of the probabilities p ij . We consider a transmission strategy which adapts the threshold values over time as specified in Section 3. Rather than using fixed user weights w i , however, we take these equal to the current queue lengths of the various users, setting w i (t) = Q i (t). Note that the transmission strategy does not use any explicit knowledge of the feasible transmission rate distributions or arrival parameters.

The above-described transmission strategy achieves stability for any arrival rate vector (λ 1, ..., λ N ) ∈ Λ. A detailed proof is beyond the scope of the present paper, and hence we only provide a brief sketch of the approach which is quite similar to that in [29], and relies on consideration of the fluid limit where the queue length process is scaled in both space and time. Specifically, we consider a sequence of queue length processes, indexed by the superscript (r), where the initial state satisfies || Q (r)(0) || = r, with r some positive scaling coefficient. For convenience, we extend the definition of the queue length process to continuous time by adopting the convention \(Q(t) = Q(\lfloor t \rfloor)\).

As observed in [29], in order to prove the ergodicity of the original discrete-time queue length process Q(t), t = 1, 2, ..., it suffices to show that there exist constants ε > 0 and τ > 0 such that for any sequence of the queue length processes Q (r)(t),
$$ \limsup_{r \to \infty} {{\mathbb E}\left\{{\frac{1}{r} || Q^{(r)}(r \tau) ||}\right\}} \leq 1 - \epsilon. $$
To establish the above property, we will consider the sequence of fluid scaled processes
$$ q^{(r)}(t) := \frac{1}{r} Q^{(r)}(r t), $$
and observe that we may impose the additional condition that the sequence of initial states converges, i.e., q (r)(0) →q(0) as r → ∞ with || q(0) || = 1.
It may be shown that any subsequence q (r)(t) has a further subsequence that converges to a trajectory q(t), referred to as a fluid sample path, which is Lipschitz continuous as function of t. For similar reasons as in [29], any fluid sample path satisfies the equations
$$ q_i(t) = (\lambda_i - T_i(t)) {{\rm I}_{\{{q_i(t) > 0}\}}}, $$
i = 1, ..., N, with
$$ T(t) \in \arg\max_{y \in {\cal T}} {\sum_{i = 1}^{N}} q_i(t) \log(y_i). $$
Since λ ∈ Λ and \({\rm closure}(\Lambda) \!=\! {\cal T}\), there exists a vector \(\mu \!\in\! {\cal T}\) with \(\lambda_i < \mu_i^*\), i = 1, ..., N. Define the Lyapunov function \(G(x) := {\sum_{i = 1}^{N}} x_i^2 / \mu_i^*\). Along similar lines as in [29], it may be shown that
$$ \frac{{\rm d}}{{\rm d}t} G(q(t)) \leq {\sum\limits_{i = 1}^{N}} q_i(t) [\lambda_i / \mu_i^* - 1], $$
which implies that \(\frac{{\rm d}}{{\rm d}t} G(q(t))\) is negative and bounded away from zero as long as G(q(t)) is positive and bounded away from zero. Hence, it follows that there exist constants ε and τ such that || q(τ) || ≤ 1 − ε for any sequence with || q(0) || = 1.

6 Numerical experiments

We now present numerical results to further investigate the convergence properties of the adaptive algorithm described in Section 4. We consider a total of three different scenarios. Throughout, the feasible transmission rates of the various users are assumed to be independent from slot to slot, with an exponential marginal distribution and possibly user-dependent means. The threshold values are always initialized to the mean transmission rates.

In Scenario I we consider a system with N = 3 users with equal weights, mean rates of 1.0, 3.0 and 5.0, respectively, and full interference, i.e., the collision probabilities are given by p ij  = 1 for all i ≠ j. We first examine the convergence of Algorithm (A). The length of the cycles is set to L k  = 1000 slots, and the coefficient in the multiplicative update is taken to be ϵ = 0.001.

Figure 1 shows the behavior of the ‘drift’ term D i (k) during the first 1,000 cycles. While the drift never reduces to zero in the presence of multiplicative updates with a constant factor, the figure confirms that the drift term cannot persistently be negative or positive. After an initial transient phase, the system rapidly settles into an equilibrium where the drift exhibits slight, random oscillations around zero.
Fig. 1

Scenario I; drift terms for Algorithm A; ϵ = 0.001

Figure 2 plots the evolution of the threshold values γ i (k) and illustrates that these converge to reasonably stable values which are proportional to the mean feasible transmission rates.
Fig. 2

Scenario I; threshold values for Algorithm A; ϵ = 0.001

Figure 3 shows the transmission probabilities \(\bar{F}_i(\gamma_i(k))\) and indicates that these are equal for the various users.
Fig. 3

Scenario I; transmission probabilities for Algorithm A; ϵ = 0.001

The above-mentioned properties of the threshold values and transmission probabilities corroborate the results described in Section 3 for the case where the feasible transmission rates of the users are exponentially distributed. In particular, for N = 3, the solution s * of Eq. 4—representing the proportionality factor between the mean rates and the threshold values—is approximately 1.472, and the corresponding activity factor is roughly 0.23.

Figure 4 plots the evolution of the threshold values during the first 100 cycles when the coefficient in the multiplicative update is increased to ε = 0.01. Because of page limitations, we henceforth only present results for the threshold values and not the drift terms and transmission probabilities. As to be expected, the convergence occurs about 10 times as fast, but the amplitude of the oscillations in equilibrium is somewhat larger.
Fig. 4

Scenario I; threshold values for Algorithm A; ϵ = 0.01

Figure 5 shows the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a variable step size δ k  = 1 / k. Thanks to the initially large and gradually reduced step size, the convergence occurs fast, and the random oscillations completely vanish in the limit, reflecting the almost-sure convergence established in Theorem 4.1.
Fig. 5

Scenario I; threshold values for Algorithm B; variable step size δ k  = 1 / k

Figure 6 shows the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a constant step size δ k  = 0.001. As a result of the constant step size, convergence occurs slower, and the amplitude of the random oscillations in equilibrium is larger.
Fig. 6

Scenario I; threshold values for Algorithm B; constant step size δ k  = 0.001

In Scenario II, we consider a system with N = 5 users with equal weights and unit mean rates. Users only partly interfere with others, and the collision probabilities (or incidence matrix of the interference graph) are given by
$$ (p_{ij}) = \left(\begin{array}{c@{\kern8pt}c@{\kern8pt}c@{\kern8pt}c@{\kern8pt}c} 0 & 0 & 1 & 0 & 1 \\ 0 & 0 & 0 & 1 & 1 \\ 1 & 0 & 0 & 1 & 1 \\ 0 & 1 & 1 & 0 & 1 \\ 1 & 1 & 1 & 1 & 0 \end{array}\right), $$
so that w  − 1 = w  − 2 = 2, w  − 3 = w  − 4 = 3, and w  − 5 = 4.
Figure 7 shows the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a variable step size δ k  = 1 / k. The solutions of Eq. 3 are now \(s_1^* = s_2^* = 1.472\), \(s_3^* = s_4^* = 1.744\), and \(s_5^* = 1.953\), respectively, which agree with the equilibrium values of the threshold parameters as observed in the figure.
Fig. 7

Scenario II; threshold values for Algorithm B; variable step size δ k  = 1 / k

Figure 8 plots the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a constant step size δ k  = 0.001. We observe again that convergence occurs slower as a result of the constant step size, and that the behavior in equilibrium is noisier.
Fig. 8

Scenario II; threshold values for Algorithm B; constant step size δ k  = 0.001

In Scenario III, we consider the same system as in Scenario II, except that after 5,000 time slots, an additional user emerges and the incidence matrix of the interference graph changes to
$$ (p_{ij}) = \left(\begin{array}{c@{\kern8pt}c@{\kern8pt}c@{\kern8pt}c@{\kern8pt}c@{\kern8pt}c} 0 & 0 & 1 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 & 1 & 0 \\ 1 & 0 & 0 & 1 & 1 & 1 \\ 0 & 1 & 1 & 0 & 1 & 1 \\ 1 & 1 & 1 & 1 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 & 0 \end{array}\right), $$
so that w  − 1  =  w  − 2  =  w  − 6  =  2 and w  − 3  =  w  − 4  =  w  − 5  =  4.
Figure 9 shows the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a variable step size δ k  = 1 / k. Once the additional user has entered the system, the solutions of Eq. 3 are \(s_1^* = s_2^* = s_6^* = 1.472\), and \(s_3^* = s_4^* = s_5^* = 1.953\), respectively, which agree with the equilibrium values of the threshold parameters in the figure. Observe however that the re-convergence after 5,000 time slots to the new equilibrium values is quite sluggish since the step size has been significantly reduced by then.
Fig. 9

Scenario III; threshold values for Algorithm B; variable step size δ k  = 1 / k

Figure 10 plots the evolution of the threshold values for Algorithm (B) with L k  = 1 slots per update and a constant step size δ k  = 0.001. Note that the algorithm exhibits better responsiveness and achieves substantially faster re-convergence to the new equilibrium values after 5,000 time slots due to the constant step size.
Fig. 10

Scenario III; threshold values for Algorithm B; constant step size δ k  = 0.001

Notes

References

  1. 1.
    Adireddy S, Tong L (2002) Optimal transmission probabilities for slotted Aloha in fading channels. In: Proc. CISS 2002Google Scholar
  2. 2.
    Adireddy S, Tong L (2005) Exploiting decentralized channel state information for random access. IEEE Trans Inf Theory 51:537–561MathSciNetCrossRefGoogle Scholar
  3. 3.
    Agrawal R, Subramanian V, Berry R (2008) Joint scheduling and resource allocation in CDMA systems. IEEE Trans Inf Theory 56(5):55–63MathSciNetGoogle Scholar
  4. 4.
    Andrews DM, Kumaran K, Ramanan K, Stolyar AL, Vijayakumar R, Whiting PA (2004) Scheduling in a queueing system with asynchronously varying service rates. Probab Eng Inf Sci 18:191–217MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Eryilmaz A, Srikant R (2005) Fair resource allocation in wireless networks using queue-length-based scheduling and congestion control. In: Proc. infocom 2005Google Scholar
  6. 6.
    Eryilmaz A, Srikant R, Perkins J (2005) Stable scheduling policies for fading wireless channels. IEEE/ACM Trans Netw 13:411–424CrossRefGoogle Scholar
  7. 7.
    Georgiadis L, Neely MJ, Tassiulas L (2006) Resource allocation and cross-layer control in wireless networks. Found Trends Netw 1:1–144CrossRefGoogle Scholar
  8. 8.
    Ghurumuruhan G, Song G, Li Y(G) (2005) Asymptotic throughput analysis of distributed multi-channel random access schemes (preprint)Google Scholar
  9. 9.
    Ghurumuruhan G, Song G, Li Y(G) (2005) Effect of multicarrier diversity on channel-aware random access schemes. In: Proc. IEEE ICC, Seoul, South Korea, pp 3637–3641Google Scholar
  10. 10.
    Ghurumuruhan G, Li Y(G), Swami A (2006) Channel-aware Aloha with imperfect CSI. In: Proc. IEEE globecomGoogle Scholar
  11. 11.
    Ghurumuruhan G, Li Y(G), Vook FW (2007) Stability region of multi-carrier channel-aware Aloha. In: Proc. IEEE WCNCGoogle Scholar
  12. 12.
    Gupta P, Stolyar AL (2006) Optimal throughput allocation in general random-access networks. In: Proc. CISS 2006Google Scholar
  13. 13.
    Kar K, Sarkar S, Tassiulas L (2003) Achieving proportionally fair rates using local information in multi-hop Aloha networks. In: Proc. 41st annual allerton conf. commun. control., compGoogle Scholar
  14. 14.
    Kar K, Sarkar S, Tassiulas L (2004) Achieving proportionally fair rates using local information in multi-hop Aloha networks. IEEE Trans Automat Contr 49:1858–1863MathSciNetCrossRefGoogle Scholar
  15. 15.
    Knopp R, Humblet PA (1995) Information theory and power control in single-cell multi-user communications. In: Proc. ICC ’95Google Scholar
  16. 16.
    Liu X, Chong EKP, Shroff NB (2003) A framework for opportunistic scheduling in wireless networks. Comput Networks 41:451–474zbMATHCrossRefGoogle Scholar
  17. 17.
    Liu J, Stolyar AL (2007) Distributed queue length based algorithms for optimal end-to-end throughput allocation and stability in multi-hop random access networks. In: Proc. 45th annual allerton conf. commun., control, compGoogle Scholar
  18. 18.
    Luo W, Ephremides A (1999) Stability of N interacting queues in random-access systems. IEEE Trans Inf Theory 45:1579–1587MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Massey J, Mathys P (1985) The collision channel without feedback. IEEE Trans Inf Theory 31:192–204MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Metivier M (1982) Semimartingales—a course on stochastic processes, De GruyterGoogle Scholar
  21. 21.
    Neely MJ, Modiano E, Li C-P (2005) Fairness and optimal stochastic control for heterogeneous networks. In: Proc. infocom 2005Google Scholar
  22. 22.
    Neely MJ, Modiano E, Rohrs CE (2002) Power and server allocation in a multi-beam satellite with time-varying channels. In: Proc. infocom 2002, pp 1451–1460Google Scholar
  23. 23.
    Neely MJ, Modiano E, Rohrs CE (2005) Dynamic power allocation and routing for time-varying wireless networks. IEEE J Sel Areas Commun 23:89–103CrossRefGoogle Scholar
  24. 24.
    Qin X, Berry R (2003) Exploiting multi-user diversity for medium access control in wireless networks. In: Proc. infocom 2003, San Francisco, CA, pp 1084–1094Google Scholar
  25. 25.
    Qin X, Berry R (2004) Opportunistic splitting algorithms for wireless networks. In: Proc. infocom 2004, Hong KongGoogle Scholar
  26. 26.
    Rao R, Ephremides A (1988) On the stability of interacting queues in a multi-access system. IEEE Trans Inf Theory 34:918–930MathSciNetCrossRefGoogle Scholar
  27. 27.
    Stolyar AL (2005) On the asymptotic optimality of the gradient scheduling algorithm for multi-user throughput allocation. Oper Res 53:12–25MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Stolyar AL (2006) Greedy primal-dual algorithm for dynamic resource allocation in complex networks. Queueing Syst 54:203–220MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Stolyar AL (2008) Dynamic distributed scheduling in random access networks. J Appl Probab 45:297–313MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Tassiulas L, Ephremides A (1993) Dynamic server allocation to parallel queues with randomly varying connectivity. IEEE Trans Inf Theory 39:466–478MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Venkitasubramaniam P, Adireddy S, Tong L (2003) Opportunistic Aloha and cross-layer design for sensor networks. In: Proc. MILCOM 2003Google Scholar
  32. 32.
    Williams D (1989) Probability with Martingales. Cambridge University PressGoogle Scholar
  33. 33.
    Wang X, Kar K (2004) Distributed algorithms for max-min fair rate allocation in Aloha networks. In: Proc. 42nd annual allerton conf. commun., control., compGoogle Scholar
  34. 34.
    Wang X, Kar K (2005) Cross-layer rate control for end-to-end proportional fairness in wireless networks with random access. In: Proc. MobiHoc ’05Google Scholar
  35. 35.
    Yu Y, Giannakis GB (2004) Opportunistic access in slotted Aloha adapted to decentralized CSI. In: Proc. globecom 2004Google Scholar
  36. 36.
    Yu Y, Giannakis GB (2006) Opportunistic medium access for wireless networking adapted to decentralized CSI. IEEE Trans Wirel Commun 5:1445–1455CrossRefGoogle Scholar

Copyright information

© The Author(s) 2010

Authors and Affiliations

  1. 1.Department of Electrical EngineeringKing Fahd University of Petroleum & MineralsDhahranSaudi Arabia
  2. 2.Department of Mathematics & Computer ScienceEindhoven University of TechnologyEindhovenThe Netherlands
  3. 3.Alcatel-Lucent Bell LabsMurray HillUSA

Personalised recommendations