Social welfare maximization for SRSNs using bio-inspired community cooperation mechanism

This paper considers social welfare maximization for spatial resource sharing networks (SRSNs), in which multiple autonomous users are spatially located and mutual influence only occurs between nearby users. To cope with a lack of central control and the restriction that only local information is available, a spatial resource sharing game is proposed. However, individual selfishness in traditional game models generally leads to inefficiency and dilemmas. Inspired by local cooperative behavior in biological systems, a community cooperation mechanism (CCM) is proposed to improve the efficiency of the game. Specifically, when a user makes a decision, it maximizes the aggregate payoffs for its local community rather than selfishly consider itself. It is analytically shown that with the bio-inspired CCM, the social optimum of SRSNs is achieved with an exchange of local information. The proposed bio-inspired CCM is very general and can be applied to various communication networks.

Distributed decision-making in wireless communication systems is a research topic that is currently attracting a lot of attention, e.g., distributed power control [1], multiple access control [2], and distributed channel selection [3]. While most existing work is limited to one-hop networks, in which the decision of one user influences all other users, this is not always true in practice. Thus, a more practical network model, in which the users are spatially located and mutual influence only emerges between neighboring users, has recently begun to draw attention. The most representative work is graphical game formulation for distributed channel selection in cognitive radio networks [4] and spatial congestion game formulation for spectrum sharing [5]. In this paper we will call this kind of network spatial resource sharing networks (SRSNs). Notably, SRSNs are characterized by the following features: (i) lack of central control, (ii) the availability of local information rather than global information, and (iii) mutual influence only emerges between neighboring users. Although SRSNs fit practical systems well, such networks are still in their infancy. Most importantly, the lack of central control and the restrictive availability of only local information create the challenging task of obtaining social optimum [6]-a problem that has not yet been considered in previous literature.
The most promising solution to cope with this lack of central control appears to be game theory [7]. Game theory is a powerful tool used to analyze interactions among multiple autonomous users, and it has been widely applied to wireless communication systems. However, players in games are generally assumed to be selfish; as a result, the outcome of the game, e.g., Nash equilibrium (NE), is generally inefficient. This is referred to as the tragedy of commons [8] and is the inherent limitation of game models with respect to their applications in wireless communication systems. Thus, recent literature includes several approaches to improve the efficiency of game models, and current methods include using a coordinate game [9], pricing [10], and bargaining [11]. However, these methods require global information regarding all other players in terms of actions and payoffs after each play. Therefore, existing methods cannot be applied to SRSNs and we need to seek for a new method that can achieve a social optimum for SRSNs using local information.
Bio-inspired approaches have been proposed to characterize the dynamics of multi-user interactions among communication devices, e.g., human behavior-inspired design for cognitive radio networks [12], evolution algorithms for spectrum allocation [13], and bio-inspired learning technologies for random access networks [14,15]. It has been shown that bio-inspired solutions achieve desirable outcomes and provide meaningful explanations from a biological perspective.
In this paper, motivated by the local cooperative behavior in biological systems, we propose a community cooperation mechanism (CCM) to achieve social optimums for SRSNs.
The key idea of the bio-inspired CCM is that when a user makes a decision, it maximizes the aggregate payoffs for its local community rather than selfishly consider itself. It is analytically shown that with the proposed bio-inspired CCM, social optimums for SRSNs are achieved with local information exchanges. To the best of our knowledge, this is the first investigation on achieving social optimums for SRSNs with local information.
The remainder of the paper is organized as follows. In section 1, a general description of spatial resource sharing networks (SRSNs) is presented. In section 2, the bio-inspired community cooperation mechanism (CCM) is proposed and its optimality is validated. In section 3, an example in wireless ad hoc networks is studied to show the optimality of bio-inspired CCM. Finally, we present a discussion and future work in section 4, with our conclusions in section 5.

Spatial resource sharing networks
In spatial resource sharing networks (SRSNs), there are N users autonomously competing for M resources. The user set is denoted as  ={1,...,N} and the resource set as . Moreover, we are given an undirected graph G=(V, E), where V is the vertex set and E is the edge set. Each vertex on G corresponds to an autonomous user. If two users, say i and j, are connected by an edge, i.e., (i, j) E, it means that they can exchange information directly and their decisions affect each other. The connection topology is completely arbitrary (i.e., it may include loops and cycles). Furthermore, each user has information regarding only its local connection topology. J n denotes the set of connected users of user n, i.e., In SRSNs, the payoff for a user is determined by its selected resource as well as the action profile of the connected users. Suppose that user n chooses action  n n a  , then it receives the following payoff: where f n () is the spatial resource sharing function, and r na n is the perceived state of resource a n by user n. Note that the form of the spatial resource sharing function needs not be identical for the users; instead, it can be user-dependent. Definition 1 Social welfare: social welfare for a SRSN is defined as the aggregate payoffs of all the users, i.e., We highlight the following distinctive features of the considered SRSNs: (1) Heterogeneous available action set, i.e., the available action set can vary from user to user.
(2) Heterogeneous perceived resource state, i.e., different users may perceive different states of the same resource.
(3) User-dependent resource sharing function, i.e., the received payoff is not only associated with the selected resource and the action profile, but also dependent on the user type.
Then, the network-centric goal is to obtain its social optimum. However, solving such a combinatorial optimization problem is NP-hard even in a centralized manner with global information [6]. Thus, the lack of central control and the restrictive availability of only local information will make the task of obtaining social optimum challenging. Therefore, we seek for a distributed approach with lower complexity in the following.

Bio-inspired community cooperation mechanism 2.1 Motivation
A lack of central control motivates us to address the resource sharing problem of SRSNs via the formulation of a game, denoted by ( , , ) , where the graph G specifies the player (user) set and the mutual influence among users, n  is the available action set of player n, and u n is the utility function of user n. The goal of each player in the game is to maximize its individual utility. Thus, the spatial resource sharing game is expressed as follows: Although the above game formulation accurately describes the interactions among users, individual selfishness generally leads to inefficiency and dilemmas [7]. Thus, improving the efficiency of the game is a key concern. However, because of the lack of central control and the restrictive availability of only local information, achieving social optimums for SRSNs is a challenging task.
We then look to bio-inspired mechanisms to solve this problem. Specifically, we resort to local cooperation, which is ubiquitous in nature, ranging from single cell [16] to groups of animals [17]. In addition, it is the foundation of human society [18]. Individuals in biological systems are willing to help others to whom they are connected [17], rather than behave selfishly. Inspired by this idea, we propose a bio-inspired community cooperation mechanism (CCM), with which each player helps all players in its local community. First, let us define the local community that is central to the bio-inspired CCM.
Definition 3 Local community: local community C n of user n is a set of users, which consists of itself and all of its connected users, i.e., { }   n n C n J . Here, an example is shown in Figure 1. The local communities of users 2 and 3 are given by C 2 ={1,2,4,7} and C 3 ={1,3,4}, respectively. Note that the different local communities of different users may partially overlap.
It can be seen that the local community is a collection of users that can exchange information directly-cooperation among the users in a local community is feasible. Thus, we relax individual selfishness in game models and consider user cooperation in a local community. This motivates us to define the utility function as follows: where J n is the connected user set of user n specified by eq.
(1) and R na n is the individually received payoff of user n specified by eq. (2). Note that the above-defined utility function is the aggregate payoffs of the local community C n . In other words, when a user makes a decision, it considers its local community rather than selfishly considers itself.

Analysis of NE
Here we define the Nash equilibrium (NE), which is the steady outcome of non-cooperative game models, and then investigate its properties.
Instinctively, the proposed bio-inspired CCM would perform well, because each user helps all the members in its local community. The theoretical results are as follows.
Theorem 1.  is an exact potential game that has at least one pure strategy NE, and the social optimum constitutes a pure strategy NE of .
where na n R is the payoff for user n specified by eq.(2).
Note that the defined potential function is exactly equal to social welfare S. Therefore, we have: That is, social optimum coincides with the global maximum of the potential function.
Suppose that an arbitrary player n unilaterally changes its channel selection from a n to n a , the change in its individual utility function caused by this unilateral change is then given by: where J k a represents the channel selection profile of the connected users set of user k, after the unilateral change of user n. At the same time, the change in the potential function caused by this unilateral change is given by: Based on eqs.
That is, the change in individual utility function caused by any player's unilateral deviation is the same as the change in the potential function. Thus, according to the definition given in [16], it is known that  is an exact potential game with the social welfare S serving as the potenial function.
Exact potential game is a unique type of potential game, and exhibits several attractive properties; the two most important properties are as follows: (1) Every potential game has at least one pure strategy NE.
(2) Any global or local maxima of the potential function constitutes a pure strategy NE.
Thus, according to eq. (9), Theorem 1 is proved. Theorem 1 states that the optimal NE of the spatial resource sharing game coincides with social optimum. However, multiple NE points normally exist in  and some of them may be suboptimal [7]. Thus, we require a learning algorithm to achieve optimal NE.

Achieving social optimum with local information
There are large number of learning algorithms capable of achieving pure strategy NE of potential games, e.g., best response dynamic [19], no-regret learning [3,20] and fictitious play [21]. However, a main drawback of the above learning algorithms is that there is no guarantee that they will converge to the optimal NE. Namely, they may converge instead to some sub-optimal NE points.
It has been shown that there is a learning algorithm, spatial adaptive play (SAP) [22], which converges to a pure NE of a potential game that maximizes the potential function with an arbitrary higher probability. To characterize SAP, we extend  to a mixed strategy form. The mixed strategy for player n at iteration k is denoted by probability distribution ( ) ( )   n n q k  , where ( )  n  denotes the set of probability distributions over action set n  . In SAP, only one player is randomly selected to update its action according to the mixed strategy, while all other players repeat their actions. This process is repeated until the stop criterion, e.g., the maximum number of iteration is achieved, is met. Formally, the SAP procedure can be described as follows. Spatial adaptive play (SAP): Initially, each user  n  randomly selects an action a n (0) from its available action set n  with equal probability. In iteration k, just one player, say i, is randomly selected. Then, it chooses an action where the a i th component ( 1)  a i i q k of the mixed strategy is given by: for some learning parameter  > 0. In the meantime, all the other players repeat their actions, i.e., where  () is the potential function.
Proof: Similar lines given in [9,22] can be applied to prove this proposition.
Theorem 2. With a sufficiently large , the bio-inspired CCM converges to the social optimum with an arbitrary high probability.
Proof: According to eq. (9), the global maximum value of the potential function can be obtained by max Thus, set    , which yields the following: Based on eqs. (15) and (16), the following can be obtained: Hence, Theorem 2 is proved. According to Theorems 1 and 2, social optimum is achieved solely via a local information exchange. Specifically, as stated in Theorem 1, social optimum constitutes a NE point of the game by using the bio-inspired CCM. Thus, as stated in Theorem 2, social optimum, i.e., the optimal NE, is achieved distributively by using SAP. To summarize, a bio-inspired CCM is desirable for SRSNs because social optimum is reached as a result of the interactions of individual decisions. From a biological perspective, this result is straightforward and elegant.
When SRSNs shrink into one-hop systems, e.g., a wireless mesh network [9], in which each user is connected to all other users, the proposed CCM is reduced to a global cooperative mechanism. In this case, social optimum also occurs as a result of individual decisions. For a detailed description of global cooperative mechanisms, refer to [9].

Algorithm discussion
In this subsection, we discuss some key concerns with regard to the application of SAP in practice.
First, there are several approaches noted in previous literature regarding the implementation of a random selection of users in distributed networks. For instance, random token passing as discussed in [9], and 802.11 DCF-based negotiation mechanism proposed in [20].
Second, the learning parameter  balances the trade-off between exploration and exploitation. Specifically, according to the updating rule specified by (14), smaller  means that the user is more likely to select a sub-optimal action, while a larger  means that it is more likely to select a better action. In particular,   = 0 means that user i will select any action a i (k+1) i A with equal probability, while    means that it will select the best action. Therefore, it is ad-visable that during the beginning phase, the value of  is set to a small number, and that it increases as the learning algorithm iterates [9].
Third, it should be pointed out that the arbitrary high probability in Theorem 2 means that the probability is sufficiently close to 1. For example, suppose that there are multiple NE points in a SRSN, which lead to the following social welfare, S={1.2,1.1,1.1,1}. P c denotes the probability that the bio-inspired CCM converges to the maximum social welfare, i.e., S max =max{S}=1.2. Then, according to Proposition 1, it is known that P c =0.9643 for  = 40. Furthermore, when the value of  increases, P c increases accordingly and is sufficiently close to 1, e.g., P c =0.9950 for  = 60 and P c =19×10 5 for  = 100.

Example of distributed channel selection in wireless ad hoc networks
The problem of distributed channel selection in a wireless ad hoc network can be formulated as a SRSN. In such a network, the available channels represent the resource set, the perceived state of a resource corresponds to the instantaneous channel characteristics, e.g., signal-to-noise ratio (SNR), and the spatial resource sharing function can be derived from the used transmission mechanisms, e.g., slotted or non-slotted Aloha, various versions of carrier sensing multiple access (CSMA), or time division multiple access (TDMA).
In the simulation study we consider the perfect CSMA transmission mechanism. The deployment of the simulated SRSN is shown in Figure 3, where each circle represents a transmitter-receiver pair and the dashed lines represent the influences among the pairs. Each transmitter-receiver pair is interchangeably referred to as a user. For simplicity, it is assumed that there are three homogeneous channels with a  unit rate. I n (a n ,a J n )={m  C n : a m =a n } represents a user set that belongs to the local community C n and also chooses action a n . Thus, the expected received payoff for each user is given by R n =1/|I n (a n ,a J n )|, where |·| represents the cardinality of a set.
The convergence behavior of the bio-inspired CCM is shown in Figure 4. Social optimum is obtained using an exhaustive search in a centralized manner. On the other hand, SAP is applied for the bio-inspired CCM. As the updating user in each iteration is selected autonomously and randomly, the global iteration number is unobservable by the users. In the simulation study, the learning parameter for each user is set to  = 1+0.225k, where k is the index number that the user has selected to update its action. Note that learning parameter  is determined distributively and is userdependent. In addition, it is noted from the figure that the bio-inspired CCM catches up with social optimum in approximately 45 iterations. This result validates the optimality of the proposed bio-inspired CCM.
In addition, we also simulated the performance of an individual interested model, where each user maximizes its individual payoff R na n (a n ,a J n ) as specified by (2). As SAP is only suitable for potential games and the individual interested model is no longer a potential game, another wellknown learning algorithm, no-regret learning [20], is applied for the individual interested model. For comparison fairness, the no-regret learning also starts from the same initial channel selection profile as that used in bio-inspired CCM. It is also noted in Figure 4 that the individual interested model converges to a local maxima of social welfare, i.e., it suffers from the tragedy of commons [8], as do most traditional game models.

Discussion and future work
The results presented in this paper are very general because we do not specify any network topology nor any form of spatial resource sharing function. Thus, the results can be applied to various scenarios where the autonomous users are spatially located and mutual influence only emerges between neighboring users. This is a promising approach that could be used to obtain optimal design in opportunistic spectrum access for cognitive radio ad hoc networks [3,23] and distributed spectrum management for wireless region area network [24].
As the proposed bio-inspired CCM is newly established, there still remain some problems that require further attention. Specifically, the following challenges should be considered in future work.
(1) The impact of random communication link failure, which is caused by deep channel fading or moving objects between neighboring users, requires consideration. In this scenario, the convergence and optimality of the SAP in the presence of random link failure needs to be re-investigated.
(2) Further investigation is required into asymmetric influence and communication between users. In some practical systems, mutually influenced users may be not able to exchange information directly. An example is a wireless local area network, in which the interference range is generally larger than the communication range.
(3) New learning algorithms should be developed. Although there is no need to collect information at a central point, an information exchange is still required in the local community. For some resource-limited networks, frequent information exchange may lead to unsustainable overheads. Thus, new learning algorithms, which require less information exchange or can converge rapidly, are more preferable. Specifically, an algorithm that requires no information about other users is desirable.
(4) The motivation behind the cooperation of the users must be considered. We have assumed that all the users are willing to help their connected users in this paper. However, this may be not true in practice. Specifically, let us suppose that a user behaves selfishly while all other users behave altruistically. In this scenario, the selfish user may receive a higher payoff than those behaving altruistically. Therefore, to ensure that the bio-CCM is robust, it is desirable that the mechanism also motivates the users to cooperate.

Conclusion
In this paper we considered social welfare maximization for spatial resource sharing networks (SRSNs), in which multiple autonomous users are spatially located and mutual influence only emerges between neighbors. Inspired by local cooperation behaviors in biological systems, a community cooperative mechanism was proposed. It is analytically shown that with the proposed mechanism, the social optimum of SRSNs is achieved with local information exchange. The results presented in this paper provide a better under-standing for distributed decision-making problems and can be applied to several communication systems. Future work focusing on more practical system models is ongoing.