Non-cooperative queueing games on a network of single server queues

Abstract

This paper introduces non-cooperative games on a network of single server queues with fixed routes. A player has a set of routes available and has to decide which route(s) to use for its customers. Each player’s goal is to minimize the expected sojourn time of its customers. We consider two cases: a continuous strategy space, where each player is allowed to divide its customers over multiple routes, and a discrete strategy space, where each player selects a single route for all its customers. For the continuous strategy space, we show that a unique pure-strategy Nash equilibrium exists that can be found using a best-response algorithm. For the discrete strategy space, we show that the game has a Nash equilibrium in mixed strategies, but need not have a pure-strategy Nash equilibrium. We show the existence of pure-strategy Nash equilibria for four subclasses: (i) N-player games with equal arrival rates for the players, (ii) 2-player games with identical service rates for all nodes, (iii) 2-player games on a $2\times 2$-grid, and (iv) 2-player games on an $A\times B$-grid with small differences in the service rates.

1 Introduction

In this paper, we introduce and analyze a new type of queueing games: non-cooperative games on a network of single server queues with fixed routes. Multiple players have to send their customers from a (player-specific) source node to a (player-specific) sink node. The players compete to minimize the expected sojourn time of their customers, and decide on a routing strategy through the network to achieve this goal. This setting may arise, for example, in a manufacturing system with duplicate machines, in which different product groups that require a subset of production lines compete over machines shared by different lines to achieve their product completion deadlines. First, we consider games with continuous strategy spaces, where a player may split its customers over several routes available to the player. Second, we consider games in which each player is only allowed to use a single route for all its customers.

For the continuous strategy space, we show that a unique pure-strategy Nash equilibrium exists that can be found using a best-response algorithm. We show that the game with discrete strategy space is a special variant of a weighted congestion game [14], which implies that the game has a Nash equilibrium in mixed strategies. However, the game need not have a Nash equilibrium in pure strategies. We show the existence of pure-strategy Nash equilibria for four subclasses: (i) N-player games with equal arrival rates for the players, (ii) 2-player games with identical service rates for all nodes, (iii) 2-player games on a $2\times 2$-grid, and (iv) 2-player games on an $A\times B$-grid with small differences in the service rates.

Our paper fits in the strategic queueing literature. The classical paper by Bell and Stidham [4] considers the allocation of customers to multiple servers to optimize expected waiting costs. Our work is more general in that we consider routes (sets of servers) that may intersect instead of parallel servers. A broad overview of models of rational behavior in queueing systems is provided in [9, 10]. Chapter 8 in [9] cites papers about routing on parallel servers. However, none of these papers concern routes on sets of servers, as we do. In the book [23], the author discusses how the preferences of the users affect the decisions on parameter values in queueing systems. Chapter 8 in [23], on multiclass networks of queues, is related to our paper. Differences are that we use another objective function and hence have different conditions on the existence of a Nash equilibrium, and in case of discrete strategy spaces we show existence of a Nash equilibrium for several games on certain types of networks. There are several papers combining models from game theory and queueing theory. For example, in communication networks, game theory is used to determine optimal routing strategies [2], and Braess’s paradox showing that adding capacity in a network may lead to higher congestion in a non-cooperative game on a queueing network is discussed in [6]. In security, queueing theory is used to model stochastic elements in interdiction games [13, 27]. A cooperative game on a network of single server queues is introduced in [24, 25], where each server in this network is considered to be a player. In [24] the players are allowed to cooperate by redistributing their service rates over the nodes in order to minimize the long-run expected queue length, whereas in [25] the players cooperate by redistributing the arrival rates of the customers.

The remainder of this paper is organized as follows: In Sect. 2, we introduce the non-cooperative game on a network of single server queues. In Sect. 3, we consider the game with a continuous strategy space in which each player is allowed to divide its customers over multiple routes. We prove the existence of a unique pure-strategy Nash equilibrium and discuss how to find this. In Sect. 4, we discuss the game with a discrete strategy space in which each player has to select a single route for all its customers. We show that a pure-strategy Nash equilibrium need not always exist and present four subclasses of games that do allow for a pure-strategy Nash equilibrium. Section 5 concludes the paper.

2 Model

In this section, we introduce our non-cooperative game on a network of single server queues with fixed routes.

Consider an open network of nodes ${\mathcal {C}}=\{1,\ldots ,C\}$, in which agents or players ${\mathcal {N}}=\{1,\ldots ,N\}$ select routes for their customers. Each node i is a First In First Out (FIFO) single server queue with exponential service at rate $\mu _i$, $i\in {\mathcal {C}}$. The arrival rate $\lambda _i$ of node i is determined by the routes chosen by the players, $i\in {\mathcal {C}}$.

The goal of each player is to minimize the mean sojourn time of its customers via a suitable partition of its customers over the routes available for the player. To this end, player $j\in {\mathcal {N}}$ has to decide how to divide its customers over a given and fixed set of routes $R^{(j)}$ that is available for this player. A route $r\in R^{(j)}$ is a sequence of nodes that are subsequently visited by the customers on this route, i.e., $r=(s(r,1),\ldots ,s(r,R_r))$, with $s(r,k) \in {\mathcal {C}}$, $k=1,\ldots ,R_r$ and $R_r$ the length of route r. Thus, a customer on route r enters the network in node s(r, 1), is served in $R_r$ nodes and leaves the network upon completion of service in node $s(r,R_r)$. The arrival process of customers for player j is a Poisson process with rate $\lambda ^{(j)}$. A strategy of player j is a vector $p^{(j)}$, where component $p^{(j)}_r$ is the fraction of its customers sent along route $r\in R^{(j)}$, so that the arrival rate of customers from player j to route r is $p^{(j)}_r\lambda ^{(j)}$.

Following [11, Section 3.1], we now introduce a FIFO single server queue serving customers of different types and a network of these FIFO queues in which customers’ routes are determined by their type.

Consider the multi-type FIFO queue i. A complete description of a FIFO queue with customer types ${\mathcal {T}}=\{1,\ldots ,T\}$ requires a description of the position of the customers in the queue as well as rules for the state change upon arrival of a new customer or a service completion. Let customers of type t arrive according to a Poisson process with rate $\lambda _i(t)$, $t\in {\mathcal {T}}$. Let the service requirement of all customers be exponential with rate $\mu _i$. Let $\rho _i(t)=\lambda _i(t) / \mu _i$, $t\in {\mathcal {T}}$. Assume that $\rho _i:=\sum _{t=1}^T \rho _i(t) < 1$. If $n_i>0$ customers are present, let $\mathbf{t}_i=(t_i(1),\ldots ,t_i(n_i))$, $t_i(a)\in {\mathcal {T}}$, record the type of the customers in position a, $a=1,\ldots ,n_i$, where the customer in position 1 is in service. If a new customer of type t arrives in state $\mathbf{t}_i=(t_i(1),\ldots ,t_i(n_i))$ the new state is $\mathbf{t}_i'=(t_i(1),\ldots ,t_i(n_i),t)$. If the customer in service completes service the new state is $\mathbf{t}_i'=(t_i(2),\ldots ,t_i(n_i))$. Let $\{M_i\}$ denote the Markov chain recording the evolution of the state of the multi-type FIFO queue at state space $\mathbf{T}_i=\{\mathbf{t}_i : \mathbf{t}_i=(t_i(1),\ldots ,t_i(n_i)),~t_i(a)\in {\mathcal {T}},~a=1,\ldots ,n_i,~n_i\in {\mathbb {N}}_0\},$ with transition rates, for $\mathbf{t}_i = (t_i(1),\ldots ,t_i(n))$, $\mathbf{t}_i' \ne \mathbf{t}_i$, $\mathbf{t}_i, \mathbf{t}_i'\in \mathbf{T}_i$,

$$\begin{aligned} q(\mathbf{t}_i,\mathbf{t}_i') =\left\{ \begin{array}{ll} \lambda _i(t) , &{}\quad \text{ if } \mathbf{t}_i'=(t_i(1),\ldots ,t_i(n_i),t),~t \in {\mathcal {T}}, \\ \mu _i , &{}\quad \text{ if } \mathbf{t}_i'=(t_i(2),\ldots ,t_i(n_i)). \end{array}\right. \end{aligned}$$

Then, $\{M_i\}$ has equilibrium distribution [11, p. 61]:

$$\begin{aligned} \pi _i(\mathbf{t}_i)= (1-\rho _i) \prod _{a=1}^n \rho _i({t_i(a)}), \quad \mathbf{t}_i=(t_i(1),\ldots ,t_i(n_i)) \in \mathbf{T}_i, \end{aligned}$$

and the mean sojourn time of a customer in this queue is $1/(\mu _i(1-\rho _i))$.

Let $p=(p^{(1)},...,p^{(N)})$ denote the strategy profile of all players and $p_{-j}=(p^{(1)},...,p^{(j-1)},$ $p^{(j+1)},...,p^{(N)})$ the strategies without player j’s strategy. The strategy profile uniquely determines the routes that are available for the customers of all players as well as the rate at which these customers arrive to the nodes along their route. To this end, let a customer of type j, r be a customer of player j that follows route r. The arrival rate of these customers to the network is $p^{(j)}_r\lambda ^{(j)}$. As a consequence, the arrival rate of these customers to each queue s(r, k), $k=1,\ldots ,R_r$, on their route r is $p^{(j)}_r\lambda ^{(j)}$. Thus, under strategy profile p, the total arrival rate for node i is

$$\begin{aligned} \lambda _i= \sum _{j=1}^N\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r\lambda ^{(j)} = \sum _{j=1}^N {\tilde{p}}_i^{(j)}\lambda ^{(j)}, \end{aligned}$$

where

$$\begin{aligned} {\tilde{p}}^{(j)}_i:=\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r \end{aligned}$$

is the fraction of customers of player j that routes via node i. Strategy profile $p=(p^{(1)},...,p^{(N)})$ is feasible if $\lambda _i < \mu _i$ for all $i\in {\mathcal {C}}$. This is the stability condition for the network of single server queues under which the equilibrium distribution for the number of customers at the nodes exists [11, p. 61]. Let

$$\begin{aligned} {\mathcal {P}} = \left\{ p\left| \sum _{\{r:r\in R^{(j)}\}}p^{(j)}_r= 1, p^{(j)}\ge 0,j\in {\mathcal {N}},\ \sum _{j=1}^N\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r\lambda ^{(j)}<\mu _i, \ i\in {\mathcal {C}}\right. \right\} \end{aligned}$$

be the set of feasible strategy profiles. Note that the strategies of the players are dependent as they are related via the stability condition $\lambda _i < \mu _i$ for all $i\in {\mathcal {C}}$.

Let $\{M\}=\{M_1,\ldots ,M_C\}$ record the evolution of the state of the Markov chain of the network of multi-type FIFO queues, with $\{M_i\}$ recording the state of queue i with arrival rate determined by the routes of the customers as described above. Then, $\{M\}$ has equilibrium distribution [11, p. 61]:

$$\begin{aligned} \pi (\mathbf{t})=\prod _{i=1}^C\pi _i(\mathbf{t}_i),\quad \mathbf{t}=(\mathbf{t}_1,\ldots ,\mathbf{t}_C) \in \mathbf{T}=\mathbf{T}_1\times \cdots \times \mathbf{T}_C. \end{aligned}$$

For a feasible strategy profile p the mean sojourn time of the customers of player j is

$$\begin{aligned} f^{(j)}(p) = \sum _{i=1}^C \frac{{\tilde{p}}^{(j)}_i}{\mu _i-\lambda _i}, \end{aligned}$$

(1)

where $\frac{1}{\mu _i-\lambda _i}$ is the expected sojourn time in node i and ${\tilde{p}}^{(j)}_i$ is the fraction of customers of player j that routes via this node.

Recall that the goal of each player is to minimize the mean sojourn time of its customers via a suitable partition of its customers over the routes available for the player. To obtain the optimal strategy profile for the players, the relevant solution concept in this case is the so-called generalized Nash equilibrium [7]. This concept has been studied a.o. in [20], whose result we will use later on. Therefore, in the sequel we refer to the equilibrium as the Nash equilibrium as in [20].

Let $\Gamma$ denote the N-player non-cooperative game on the network of multi-type FIFO single server queues, or ’game’ in short. This game is defined by the tuple $({\mathcal {N}},\{S^{(j)},f^{(j)}\}_{j\in {\mathcal {N}}})$, where ${\mathcal {N}}$ is the set of players, $S^{(j)}$ is the set of all strategies $p^{(j)}$, as defined by ${\mathcal {P}}$, of player j and $f^{(j)}$ is player j’s objective function as given in (1). In the game $\Gamma$, the players choose their strategies simultaneously. The objective of each player is to minimize the mean sojourn time of its customers.

A strategy $p^{(j)}$ is called dominant if it results in the lowest mean sojourn time for player j no matter the strategies chosen by the other players: $f^{(j)}(p^{(j)},p_{-j})\le f^{(j)}({\bar{p}})$ for all $p_{-j}$, ${\bar{p}}$. A strategy profile p is a Nash equilibrium if no player can decrease its expected sojourn time by unilateral deviation: for each player j, $f^{(j)}(p^{(j)},p_{-j})\le f^{(j)}({\bar{p}}^{(j)},p_{-j})$ for all ${\bar{p}}^{(j)}$.

We will consider discrete and continuous strategy spaces. In the basic form, a player is only allowed to select a single route for its customers given the feasible strategy $p_{-j}$ of the other players. This gives rise to discrete and finite strategy spaces for the players. A strategy profile p is a pure-strategy profile if

$$\begin{aligned} \left\{ \begin{array}{lll} p^{(j)}_r\in \{0,1\}, &{} r\in R^{(j)},~j\in {\mathcal {N}},&{} \text{ discrete } \text{ strategy } \text{ space },\\ p^{(j)}_r\in [0,1], &{} r\in R^{(j)},~j\in {\mathcal {N}},&{} \text{ continuous } \text{ strategy } \text{ space }, \end{array} \right. \end{aligned}$$

i.e., in the case of a discrete strategy space each player selects a single route for all its customers, and in the case of a continuous strategy space each player may partition its customers over the routes available for the player. Such a strategy profile p is called a pure-strategy profile to distinguish it from the mixed strategies that we will introduce in Sect. 4.

For the discrete case, the optimal strategy for player j can be found by solving the following mathematical program given the strategies of the other players:

$$\begin{aligned} V_d^{(j)}=\min _{p^{(j)}}\quad&\sum _{i=1}^C \frac{{\tilde{p}}_i^{(j)}}{\mu _i-\lambda _i} \end{aligned}$$

(2)

$$\begin{aligned} \text {s.t. }&\lambda _i = \sum _{k=1}^N{\tilde{p}}_i^{(k)}\lambda ^{(k)},\quad&i\in {\mathcal {C}}, \end{aligned}$$

(3)

$$\begin{aligned}&\lambda _i < \mu _i,&i\in {\mathcal {C}}, \end{aligned}$$

(4)

$$\begin{aligned}&\sum _{\{r:r\in R^{(j)}\}} p_r^{(j)}=1,&\end{aligned}$$

(5)

$$\begin{aligned}&p^{(j)}_r\in \{0,1\},&r \in R^{(j)}. \end{aligned}$$

(6)

To obtain the optimal strategy profile p, the pure-strategy Nash equilibrium of the game, we solve these problems simultaneously for all players.

In the following section, we first consider continuous strategy spaces, i.e., we relax the restriction (6) to $p^{(j)}_r\in [0,1]$, which allows a player to partition its customers over the routes available to the player. In Sect. 4 we return to discrete strategy spaces and investigate both the mixed-strategy case, where a player may randomize over a selection of routes and pure strategies, where each player sends all its customers over a single route available to the player, i.e., the game under restriction (6).

3 Continuous strategy space

This section considers the game $\Gamma$ with a continuous strategy space, in which each player is allowed to partition its customers over the routes available to the player. We show that there exists a unique pure-strategy Nash equilibrium for this game and discuss approaches to find this Nash equilibrium.

The objective of each player is to minimize the mean sojourn time of its customers. Consider player j. Given the strategy $p_{-j}$ of the other players, the optimal strategy for player j can now be found by solving the following mathematical program:

$$\begin{aligned} V_c^{(j)} = \min _{p^{(j)}}\quad&\sum _{i=1}^C \frac{{\tilde{p}}_i^{(j)}}{\mu _i-\lambda _i} \end{aligned}$$

(7)

$$\begin{aligned} \text {s.t. ~ }&\lambda _i = \sum _{k=1}^N{\tilde{p}}_i^{(k)}\lambda ^{(k)},\quad&i\in {\mathcal {C}}, \end{aligned}$$

(8)

$$\begin{aligned}&\lambda _i < \mu _i,&i\in {\mathcal {C}}, \end{aligned}$$

(9)

$$\begin{aligned}&\sum _{\{r:r\in R^{(j)}\}} p_r^{(j)}=1,&\end{aligned}$$

(10)

$$\begin{aligned}&p^{(j)}_r\in [0,1],&r \in R^{(j)} , \end{aligned}$$

(11)

which is obtained from (2)–(6) by replacing (6) by its relaxation $p^{(j)}_r\in [0,1]$ in (11). Obviously, given the strategies $p_{-j}$ chosen by the other players $V_c^{(j)} \le V_d^{(j)}$.

Solving (7)–(11) simultaneously for all players j results in the optimal strategy profile p that is the Nash equilibrium of the game $\Gamma$. To this end, the method of Lagrange multipliers may be used. As the example below shows, this is tractable for small networks, only.

Example 1

Consider a network with three nodes and two players with sets of routes $R^{(1)} = \{\{1\},\{2\}\}$ and $R^{(2)} = \{\{2\},\{3\}\}$. Assume that $\lambda ^{(1)}<\mu _1$, $\lambda ^{(2)}<\mu _3$ and $\lambda ^{(1)}+\lambda ^{(2)}<\mu _2$, which guarantees stability for all nodes. The payoff functions for players 1 and 2 are

$$\begin{aligned}&f^{(1)}(p) = \frac{p^{(1)}_{1}}{\mu _1-p^{(1)}_{1}\lambda ^{(1)}} + \frac{1-p^{(1)}_{1}}{\mu _2-(1-p^{(1)}_{1})\lambda ^{(1)}-(1-p^{(2)}_{3})\lambda ^{(2)}}, \\&f^{(2)}(p) = \frac{p^{(2)}_{3}}{\mu _3-p^{(2)}_{3}\lambda ^{(2)}} + \frac{1-p^{(2)}_{3}}{\mu _2-(1-p^{(1)}_{1})\lambda ^{(1)}-(1-p^{(2)}_{3})\lambda ^{(2)}}, \end{aligned}$$

respectively, where $p^{(j)}_{i}$ is the fraction of customers that player j sends along the route through node i, and $1-p^{(j)}_{i}$ the fraction that this player sends along the other route by (10).

The Lagrangians for the players are

$$\begin{aligned} L_1(p^{(1)},\alpha _1,\alpha _2)&= f^{(1)}(p)+ \alpha _1p^{(1)}_{1}+\alpha _2(1-p^{(1)}_{1}),\\ L_2(p^{(2)},\beta _1,\beta _2)&=f^{(2)}(p)+\beta _1p^{(2)}_{3}+\beta _2(1-p^{(2)}_{3}). \end{aligned}$$

The optimal solution for $p^{(1)}_{1}$ and $p^{(2)}_{3}$ can now be found from the Karush–Kuhn–Tucker conditions:

$$\begin{aligned} \frac{\partial L_1}{\partial p^{(1)}_{1}}&= \frac{\mu _1}{(\mu _1-p^{(1)}_{1}\lambda ^{(1)})^2} - \frac{\mu _2-(1-p^{(2)}_{3})\lambda ^{(2)}}{(\mu _2-(1-p^{(1)}_{1})\lambda ^{(1)}-(1-p^{(2)}_{3})\lambda ^{(2)})^2}+\alpha _1-\alpha _2 = 0,\\ \frac{\partial L_2}{\partial p^{(2)}_{3}}&= \frac{\mu _3}{(\mu _3-p^{(2)}_{3}\lambda ^{(2)})^2} - \frac{\mu _2-(1-p^{(1)}_{1})\lambda ^{(1)}}{(\mu _2-(1-p^{(1)}_{1})\lambda ^{(1)}-(1-p^{(2)}_{3})\lambda ^{(2)})^2}+\beta _1-\beta _2 = 0,\\ \alpha _1p^{(1)}_{1}&=0,\\ \alpha _2(1-p^{(1)}_{1})&=0,\\ \beta _1p^{(2)}_{3}&=0,\\ \beta _2(1-p^{(2)}_{3})&=0,\\ 0\le p^{(1)}_{1}&\le 1,\\ 0\le p^{(2)}_{3}&\le 1. \end{aligned}$$

Solving this system gives the optimal solution. As an illustration, for $\mu _1 = \mu _3 = 3,~ \mu _2 = 4$, $\lambda ^{(1)}=\lambda ^{(2)}=1$, the optimal solution satisfies

$$\begin{aligned} \left\{ \begin{array}{l} 3(2+p^{(1)}_{1}+p^{(2)}_{3})^2 = (3-p^{(1)}_{1})^2 (3+p^{(2)}_{3}),\\ 3(2+p^{(1)}_{1}+p^{(2)}_{3})^2 = (3-p^{(2)}_{3})^2 (3+p^{(1)}_{1}) . \end{array}\right. \end{aligned}$$

By symmetry $p^{(1)}_{1}= p^{(2)}_{3}$, resulting in

$$\begin{aligned} p^{(1)}_{1}= p^{(2)}_{3} = 5 + \frac{9 i (\sqrt{3} + i)}{\root 3 \of {25 + 2 i \sqrt{26}}} + (-1 - i \sqrt{3}) \root 3 \of {25 + 2 i \sqrt{26}} \approx 0.39, \end{aligned}$$

(12)

which satisfies the restriction (9).

As illustrated in Example 1, we may use the method of Lagrange multipliers and the Karush–Kuhn–Tucker conditions to find a Nash equilibrium for the general game $\Gamma$. However, this is intractable for larger networks. Another approach to find Nash equilibria is to use a best-response approach where all players iteratively implement their best-response strategy [17].

The best response of player j to the other players’ strategies $p_{-j}$ is represented by the set $B^{(j)}(p_{-j}) = \{{\hat{p}}^{(j)}\in S^{(j)}| f^{(j)}({\hat{p}}^{(j)},p_{-j})\le f^{(j)}({\bar{p}}^{(j)},p_{-j})\ \forall {\bar{p}}^{(j)}\}.$ In particular, p is a pure-strategy Nash equilibrium if each player’s strategy is a best response to the other players’ strategies: $p^{(j)}\in B^{(j)}(p_{-j})$ for all j. As is shown in Theorem 1, if ${\mathcal {P}}$ is non-empty, this approach yields a pure-strategy Nash equilibrium. The proof of this result requires the following two Lemmas.

First, we show that the strict inequality $\lambda _i < \mu _i$ in the mathematical program (7)–(11) may be replaced by a weak inequality $\lambda _i \le \mu _i$, which yields the mathematical program

$$\begin{aligned} {\bar{V}}_c^{(j)} = \min _{p^{(j)}}\quad&\sum _{i=1}^C \frac{{\tilde{p}}_i^{(j)}}{\mu _i-\lambda _i} \end{aligned}$$

(13)

$$\begin{aligned} \text {s.t. ~ }&\lambda _i = \sum _{k=1}^N{\tilde{p}}_i^{(k)}\lambda ^{(k)},\quad&i\in {\mathcal {C}}, \end{aligned}$$

(14)

$$\begin{aligned}&\lambda _i \le \mu _i,&i\in {\mathcal {C}}, \end{aligned}$$

(15)

$$\begin{aligned}&\sum _{\{r:r\in R^{(j)}\}} p_r^{(j)}=1,&\end{aligned}$$

(16)

$$\begin{aligned}&p^{(j)}_r\in [0,1],&r \in R^{(j)} . \end{aligned}$$

(17)

Relaxing the strict inequalities (9) to weak inequalities (15) adds only solutions with infinite objective values to the feasible region, which does not change the set of optimal solutions. This is formalized in the following lemma.

Lemma 1

A solution that is optimal for the mathematical program (7)–(11) is also optimal for (13)–(17), and vice versa.

Proof

Observe that the strategy $p_{-j}$ of the other players is fixed both in (7)–(11) and in (13)–(17).

Let $p^{*(j)}$ denote an optimal solution of the mathematical program (7)–(11). This solution results in a finite value of the objective function. This solution is also a feasible solution of the program (13)–(17). But then it is also optimal for this latter program since if node i is used an optimal solution will not have $\lambda _i=\mu _i$, $i\in {\mathcal {C}}$, as this would result in an infinite value of the objective function, and if node i is not used by any of the players for this node i the restriction on $\lambda _i$ is redundant in both (7)–(11) and (13)–(17).

We will prove the reversed statement by contradiction. Let $p^{'(j)}$ denote the optimal solution of the mathematical program (13)–(17), resulting in a finite value of the objective function. Assume $p^{'(j)}$ is not an optimal (and thus not a feasible) solution of (7)–(11). This means that for at least one $i\in {\mathcal {C}}$, $\lambda _i=\mu _i$. However, this would result in an infinite value of the objective function, which is in contradiction with the optimality of $p^{'(j)}$. Thus, $p^{'(j)}$ is also an optimal solution of (7)–(11). $\square$

Second, we show convexity of the payoff function of a player given fixed strategies of the other players.

Lemma 2

The payoff function $f^{(j)}(p)$ of player j is strictly convex in $p^{(j)}$, $j \in {\mathcal {N}}$, $p\in {\mathcal {P}}$.

Proof

We will consider the Hessian of $f^{(j)}$. To this end, we first rearrange $f^{(j)}(p)$ as a function of ${\tilde{p}}^{(j)}_i:=\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r$, the fraction of customers from player j that routes via node i. As $p_{-j}$ is fixed, the service rate that remains available at node i for customers routed by player j is ${\tilde{\mu }}_i^{(j)} = \mu _i-\sum _{k=1,~k\ne j}^N {\tilde{p}}^{(k)}_i\lambda ^{(k)}$. Then, $f^{(j)}({p})$ may be written as

$$\begin{aligned} f^{(j)}({p}) = \sum _{i=1}^{C} \frac{{\tilde{p}}_i^{(j)}}{{\tilde{\mu }}_i^{(j)}-{\tilde{p}}_i^{(j)}\lambda ^{(j)}}, \end{aligned}$$

which may be viewed as a function of ${\tilde{p}}_i^{(j)}$, $i \in {\mathcal {C}}$. Observe that

$$\begin{aligned} \frac{\partial f^{(j)}(p)}{\partial {\tilde{p}}_i^{(j)}}&= \frac{{\tilde{\mu }}_i^{(j)}}{({\tilde{\mu }}_i^{(j)}-{\tilde{p}}_i^{(j)}\lambda ^{(j)})^2},&i \in {\mathcal {C}},\\ \frac{\partial ^2 f^{(j)}(p)}{(\partial {\tilde{p}}_i^{(j)})^2}&= \frac{2{\tilde{\mu }}_i^{(j)}\lambda ^{(j)}}{({\tilde{\mu }}_i^{(j)}-{\tilde{p}}^{(j)}_i\lambda ^{(j)})^3},&i\in {\mathcal {C}}, \\ \frac{\partial ^2 f^{(j)}(p)}{\partial {\tilde{p}}_i^{(j)}\partial {\tilde{p}}_k^{(j)}}&= 0,&i,k\in {\mathcal {C}} . \\ \frac{\partial {\tilde{p}}_i^{(j)}}{\partial p_r^{(j)}}&= {\mathbb {1}}(i\in r) ,&i \in {\mathcal {C}}, \end{aligned}$$

with $\mathbb {1}(A)$ being the indicator function of event A. The Hessian of $f^{(j)}$ has the following entries, for $r,s \in R^{(j)}$:

$$\begin{aligned} \frac{\partial ^2 f^{(j)}(p)}{\partial {p}_r^{(j)}\partial {p}_s^{(j)}}&= \sum _{\{i:i\in r\cap s\}} \frac{\partial ^2 f^{(j)}(p)}{(\partial {\tilde{p}}_i^{(j)})^2} . \end{aligned}$$

We may rearrange the rows and columns of the Hessian such that they correspond with the routes in decreasing order of the route lengths, where routes of equal length may be arranged in random order. The ith diagonal element of this rearranged Hessian is always larger than or equal to all elements in row and column i. It can readily be shown that all pivots of this Hessian are positive. Thus, the Hessian is positive definite, so that $f^{(j)}(p)$ is strictly convex in $p^{(j)}$. $\square$

We are now ready to show existence of a unique pure-strategy Nash equilibrium for the game $\Gamma$ and convergence of the best-response algorithm to this pure-strategy Nash equilibrium.

Theorem 1

If ${\mathcal {P}}$ is non-empty, the non-cooperative game $\Gamma$ on a network of single server queues has a unique pure-strategy Nash equilibrium. Moreover, Algorithm 1 converges to the pure-strategy Nash equilibrium.

Proof

Let $\Gamma '$ denote the game based on the mathematical programs (13)–(17) for all players, and denote by ${\mathcal {P}}'$ the corresponding strategy space of all players.

As ${\mathcal {P}}$ is non-empty, the strategy space ${\mathcal {P}}'$ of $\Gamma '$ is also non-empty. In addition, the sets ${\mathcal {P}}$ and ${\mathcal {P}}'$ are bounded, since $\sum _{\{r:r\in R^{(j)}\}}p^{(j)}_r= 1$, and $p^{(j)}\ge 0$, and convex, since they are constructed with linear (in)equalities. Moreover, the set ${\mathcal {P}}'$ is closed.

By Lemma 2, the payoff function is convex for each player. Since $\frac{x}{\mu -x\lambda }$ is continuous for $x \in \{x|x<\lambda / \mu \}$, the payoff function is continuous on ${\mathcal {P}}'$. Theorem 1 in [20] now implies that the game $\Gamma '$ has a unique pure-strategy Nash equilibrium. By Lemma 1, given the strategies of the other players the optimal strategy for a player in the game $\Gamma '$ is also optimal for the game $\Gamma$. Hence, the pure-strategy Nash equilibrium for game $\Gamma '$ is also a pure-strategy Nash equilibrium for the game $\Gamma$. Since each optimal strategy for a player in $\Gamma$ is also optimal for $\Gamma '$ by Lemma 1, the optimal pure-strategy Nash equilibrium must be a unique pure-strategy Nash equilibrium for $\Gamma$.

In [22, Algorithm 1] a best-response algorithm for games with closed, convex strategy sets and continuous, convex payoff functions is given, which is shown to converge to a Nash Equilibrium in [18, Theorem 3]. Our game satisfies these conditions, and Algorithm 1 is a special case of this best-response algorithm [22, Algorithm 1]. Therefore, it follows that the best-response Algorithm 1 converges to the pure-strategy Nash equilibrium. $\square$

4 Discrete strategy space

In this section, we consider our model in its basic form, in which a player is allowed to select a single route for its customers, only. This game $\Gamma ''$, say, to distinguish it from the games in Sect. 3, has discrete and finite strategy spaces for the players. Recall that for a discrete strategy space, a strategy profile p is a pure-strategy profile if $\sum _{\{r:r\in R^{(j)}\}}p^{(j)}_r= 1$ and $p^{(j)}_r\in \{0,1\}$ for all $j\in {\mathcal {N}}$. The optimal pure strategy for player j is a solution of the mathematical program (2)–(6). Solving these programs simultaneously for all players results in the pure-strategy Nash equilibrium of the game.

The example below compares Nash equilibria of games with continuous and discrete strategy spaces.

Example 1

(Continued) Reconsider the game on the network of Example 1, with three nodes and two players with sets of routes $R^{(1)} = \{\{1\},\{2\}\}$ and $R^{(2)} = \{\{2\},\{3\}\}$, but now each player must select a single route for all its customers. For $\mu _1 = \mu _3 = 3,~ \mu _2 = 4$, $\lambda ^{(1)}=\lambda ^{(2)}=1$, a bimatrix with the players’ mean sojourn time of their customers for this game is:

The rows refer to the routes of player 1 and the columns to the routes of player 2, e.g., the entry (1/2, 1/3) indicates that the mean sojourn time (see (1)) for the customers of player 1 is 1/2 and the customers of player 2 is 1/3 if player 1 selects route $\{1\}$ and player 2 selects route $\{2\}$. This game has three pure-strategy Nash equilibria:

• player 1 selecting route $\{ 1\}$ and player 2 selecting route $\{ 2\}$ with mean sojourn times 1/2 and 1/3, respectively,

• both players selecting route $\{ 2\}$ with mean sojourn time of 1/2 for both players, and

• player 1 selecting route $\{ 2\}$ and player 2 selecting route $\{ 3\}$ resulting in mean sojourn times 1/3 and 1/2, respectively.

in which the players’ mean sojourn time of their customers is either 1/3 or 1/2. By definition, in each of these strategy profiles a player does not want to deviate to another route for its customers since it would not decrease the mean sojourn time of its customers. Observe that selecting route $\{2\}$ for player 1 and route $\{2\}$ for player 2 are the dominant strategies, as these routes may lead to the mean sojourn time 1/3. Note that the dominant strategies result in the Nash equilibrium with mean sojourn time 1/2 for both players.

Finally, notice that the discrete equilibria differ from the continuous equilibrium $p^{(1)}_{1} = p^{(2)}_{3} = 0.39$, $p^{(1)}_{2} = p^{(2)}_{2} =0.61$, with mean sojourn times 0.37 for the customers of players 1 and 2 in the game with continuous strategy spaces.

The game $\Gamma ''$ is a variant of a weighted congestion game. In a congestion game multiple resources are available and each player selects a subset of these resources to minimize its own costs. The costs of each resource depend on the number of players selecting that resource. In a traditional congestion game, all players are equivalent and have the same influence on the costs of a single resource. For these games, there exists a Nash equilibrium in pure strategies [21]. In a weighted congestion game each player has a weight and the costs for each resource depend on the weighted sum over the players that pick that resource. In general, weighted congestion games do not always possess a Nash equilibrium in pure strategies (cf. [8, 14]). There are several subclasses of weighted congestion games for which pure-strategy Nash equilibria do exist, for example, for matroid congestion games in which each player’s strategy space contains the bases of a matroid on the set of resources [1], for games with affine or exponential cost functions [8], and for games in which players can split their total weight (using integer values only) over the resources and the cost functions are convex and monotonically increasing [26]. For Shapley network congestion games, where the players split the costs of a shared edge pure-strategy Nash equilibria exist when at most two players can share an edge or when all players have the same source and sink node [3, 5, 12].

For an N-player game $\Gamma ''$ on a network of single server queues with a discrete strategy space, the following example shows that existence of a pure-strategy Nash equilibrium is not guaranteed.

Example 2

Inspired by [8], as depicted in Fig. 1, consider a 2-player game on a network of single server queues with six nodes and service rates $\mu _1 = \mu _2 = \mu _5 = \mu _6 = 6$ and $\mu _3 = \mu _4 = 4.95$. Both player 1 (blue) and player 2 (red) have two possible strategies: the strategies of player 1 are to select route $r^{(1)}_1 = \{1,2,3\}$ or $r^{(1)}_2 = \{4,5,6\}$, and the strategies of player 2 are to select route $r^{(2)}_1 = \{1,2,4\}$ or $r^{(2)}_2 = \{3,5,6\}$. The arrival rates are $\lambda ^{(1)}=1$ and $\lambda ^{(2)}=2$. A bimatrix representing the mean sojourn times of the customers of both players for this game is:

For each pair of strategies $(r^{(1)}_r,r^{(2)}_s)$, $r,s=1,2$, one of the players has the incentive to switch to another strategy. Therefore, this game does not have a pure-strategy Nash equilibrium.

Fig. 1

An example of a game on a 6-node network of single server queues with discrete strategy space without a pure-strategy Nash equilibrium

A pure-strategy Nash equilibrium need not exist. Therefore, we also consider mixed strategies for the players. A mixed-strategy profile q describes for each player a probability distribution over its pure strategies (routes):

$$\begin{aligned} \sum _{\{r:r\in R^{(j)}\}}q^{(j)}_r= 1,\ q^{(j)}\ge 0,\ j\in {\mathcal {N}}. \end{aligned}$$

A mixed strategy $q^{(j)}$ represents the random selection of a single route $r \in R^{(j)}$ by player j according to the specified probabilities $q^{(j)}_r$. The player will then send all its customers over this single route r. Note that a route $r\in R^{(j)}$ selected by player j must be feasible for all routes that may be selected by the other players. Hence, for mixed strategies, we have the additional feasibility condition

$$\begin{aligned} \sum _{j=1}^N\lambda ^{(j)}\mathbb {1}(\exists r:i\in r,r\in R^{(j)} )<\mu _i, \ i\in {\mathcal {C}}, \end{aligned}$$

(18)

i.e., the service rate of node i must exceed the combined load of all players that may use node i.

For completeness, we include the following Theorem about the existence of Nash Equilibria for discrete strategy spaces. This result follows directly from [16, Theorem 1]. We will then illustrate the mixed strategies via continuation of Examples 1 and 2.

Theorem 2

The game $\Gamma ''$ with discrete strategy spaces and additional feasibility condition (18) has a Nash equilibrium in mixed strategies.

In the following, we compare the mixed strategies for the game with discrete strategy space to the strategies of the game with continuous strategy space for Examples 1 and 2.

Example 2

(Continued) Reconsider the game with discrete strategies from Example 2. This game has no pure-strategy Nash equilibrium in discrete strategies. A Nash equilibrium in discrete mixed strategies is $q^{(1)}=(0.5,0.5)$, $q^{(2)}=(0.5,0.5)$, resulting in an expected payoff of 0.916 for player 1 and 1.009 for player 2. A pure-strategy Nash equilibrium in continuous strategies has an expected payoff of 0.734 for both player 1 and 2 with $p^{(1)}=(0.5,0.5)$, $p^{(2)}=(0.5,0.5)$.

Example 1

(Continued) Reconsider the game of Example 1. In the Nash equilibrium ((0.39, 0.61), (0.61, 0.39)) of the game with continuous strategy spaces, the players partition their customers over the available routes. One may wonder if such a partition could be an optimal mixed strategy in the current game with discrete strategy spaces. For this, we compute the mixed-strategy Nash equilibria of the current game [19, section 13.2], resulting in all strategy profiles q such that

• either player 1 selects route $\{ 2\}$ and player 2 randomizes over its routes, or

• player 2 selects route $\{ 2\}$ and player 1 randomizes over its actions;

or,

$$\begin{aligned} \{q=((q^{(1)}_1,q^{(1)}_2),(q^{(2)},q^{(2)}_3)| q^{(1)}_1=0 \text{ or } q^{(2)}_2=1 \}. \end{aligned}$$

(19)

in short. The corresponding expected sojourn times are $(1/3+q^{(2)}_2/6,1/2)$ if $q^{(1)}_1=0$ and $(1/2,1/2-q^{(1)}_1/6)$ if $q^{(2)}_2=1$. Notice that the pure-strategy equilibrium in Example 1 with continuous strategy spaces, ((0.39, 0.61), (0.61, 0.39)), is not a Nash equilibrium of the game with discrete strategy spaces (19), and vice versa. This motivates us to study this latter type of games.

We now return to the game $\Gamma ''$ under pure strategies. Existence of a pure-strategy Nash equilibrium is not guaranteed in general. We devote the remainder of this section to studying subclasses of N-player games that allow for a pure-strategy Nash equilibrium. First, in Theorem 3 we will consider the case of equal arrival rates for all players. Second, in Theorem 4 we consider the case of a 2-player game with equal service rates at the nodes. Third, we investigate the more general case of unequal arrival rates and unequal service rates for a 2-player game on a 2 $\times$ 2-grid in Theorem 5. Finally, we consider a 2-player game on a general network with general arrival rates and general service rates, under the restriction that the arrival rates of the players are reasonably close, and service rates at the nodes are reasonably close, as specified in Theorem 6.

If the arrival rates of customers for all players are identical, the game translates to a traditional congestion game that has a pure-strategy Nash equilibrium.

Theorem 3

(Equal arrival rates) The N-player game on a network of single server queues with equal arrival rates $\lambda ^{(j)}=\lambda$, $j\in {\mathcal {N}}$, has a pure-strategy Nash equilibrium.

Proof

We will show that the N-player game on a network of single server queues with equal arrival rates $\lambda ^{(j)}=\lambda$, $j\in {\mathcal {N}}$, is a congestion game. The result then follows since a congestion game has a pure-strategy Nash equilibrium [21].

In the congestion game introduced in [21], all players choose a finite subset of elements (in our case nodes) as a strategy, where the strategy set might be different for each player. The delay function of each element must be positive and monotone increasing in the number of players that choose that element. We will now show that our game has these properties.

Consider a strategy profile p for all players. Let $a_{i}^{(j)}(p)=1$ if node i is used by player j in profile p, and 0 otherwise. The number of players using node i is $x_i(p):=\sum _{j=1}^N a_{i}^{(j)}(p)$. Under strategy profile p, the sojourn time (1) of player j is

$$\begin{aligned} f^{(j)}(p) =\sum _{i=1}^C \frac{\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r}{\mu _i-\lambda _i}= \sum _{i=1}^C \frac{a_{i}^{(j)}(p)}{\mu _i-\lambda _i}, \end{aligned}$$

and, as $\lambda ^{(j)}=\lambda$, $j\in {\mathcal {N}}$,

$$\begin{aligned} \lambda _i= \sum _{j=1}^N\sum _{\{r:i\in r,r\in R^{(j)}\}}p^{(j)}_r\lambda ^{(j)}=\sum _{j=1}^N a_{i}^{(j)}(p)\lambda ^{(j)}=x_i(p) \lambda . \end{aligned}$$

Let the delay function $d_i:{\mathbb {N}}\rightarrow {\mathbb {R}}$ be defined as

$$\begin{aligned} d_i(x)= \frac{1}{\mu _i-x\lambda }, \end{aligned}$$

then

$$\begin{aligned} f^{(j)}(p) = \sum _{i=1}^C a_{i}^{(j)}(p) d_i( x_i(p)) . \end{aligned}$$

In its domain $\{x|x<\mu _i / \lambda \}$, $d_i(x)$ is positive and monotone increasing. Note that $x_i(p)<\mu _i / \lambda$ is required for strategy profile p to be feasible. This implies that the N-player game on a network of single server queues with equal arrival rates is a congestion game. $\square$

As illustrated in Example 2, games with different arrival and different service rates need not have a pure-strategy Nash equilibrium. Theorem 3 shows that games with equal service rates have a pure-strategy Nash equilibrium. For a 2-player game, the following theorem considers the complementary case of identical service rates at all nodes.

Theorem 4

A 2-player game on a network of single server queues with identical service rates $\mu _i=\mu$ for all nodes $i\in {\mathcal {C}}$ has a pure-strategy Nash equilibrium.

Proof

Theorem 3.12 in [8] states that there exists a pure-strategy Nash equilibrium for a 2-player weighted congestion game in which the cost function for each resource (node) can be written as $am(x)+b$, where $a,b\in {\mathbb {R}}$, and m(x) is a monotone function that is the same for all resources (nodes).

In our 2-player game we have $\mu _i=\mu$, $i\in {\mathcal {C}}$. Let $m(x) = \frac{1}{\mu -x}$, which is a monotone increasing function for $x<\mu$. For each node the cost function equals $m(\lambda _i)$ and can therefore be written as $am(x)+b$ with $a=b=1$. Thus, this game has a pure-strategy Nash equilibrium. $\square$

The following two theorems discuss 2-player games where the network is an $A \times B$-grid. The network has AB nodes that may be represented by their coordinates (x, y), $x=1,\ldots ,A,~y=1,\ldots ,B$. From node (x, y), customers may only route to the neighboring nodes $(x-1,y)$, $(x+1,y)$, $(x,y-1)$ and $(x,y+1)$ provided these nodes are part of the grid. Let $\mu _{i}$ be the service rate of node i, $i=(x,y)$, and let $\lambda ^{(j)}$ be the arrival rate of customers for player j, $j=1,2$.

Each player’s set of possible paths is the set of all routes starting at a fixed source node at one side of the grid and ending at a fixed sink node on the opposite side of the grid. Furthermore, this set only contains routes of minimal length, i.e., containing the minimum number of nodes required to move from source node to sink node. Each player’s goal is to minimize the expected sojourn time of its customers.

First we discuss the subclass of 2-player games on a $2\times 2$-grid. A trivial situation is one in which a player’s source and sink nodes are neighbors on the grid. Then, the unique optimal route for that player is to go directly from the source node to the sink node. We will omit these trivial situations in the proof of Theorem 5. Two interesting cases are shown in Fig. 2. Here, each player needs to route through three nodes and up front it is not clear if a Nash equilibrium exists and which route will be preferred. In case (a), a player has to share the opponent’s source or sink node. The key question is: Which shared node will the player prefer? In case (b), a player may share the intermediate node with the opponent. Here, the question is whether the player wants to share the intermediate node or not. The proof of Theorem 5 reveals the answers to these key questions.

Fig. 2

This figure shows two cases of a 2-player game on a 2$\times$2-grid. The nodes are named a, b, c, d. The source nodes are shown by inward arrows, with the player’s name besides the arrow; outward arrows start from the sink nodes. In subfigure a the players 1 and 2 have different source and sink nodes. In subfigure b the players share their source and sink nodes

Theorem 5

A 2-player game on a network of single server queues on a 2$\times$2-grid as depicted in Fig. 2 with service rates $\mu _a,\mu _b,\mu _c,\mu _d$ such that $\lambda ^{(1)}+\lambda ^{(2)} <\min \{\mu _a,\mu _b,\mu _c,\mu _d\}$ has a pure-strategy Nash equilibrium.

Proof

Consider a $2\times 2$-grid as shown in Fig. 2. First, assume the players have different source and sink nodes as depicted in Fig. 2a. Player 1 has to decide between routes

$$\begin{aligned} r^{(1)}_{1} = \{a,b,d\} \text{ and } r^{(1)}_{2} = \{a,c,d\} \end{aligned}$$

for its arrivals with rate $\lambda ^{(1)}$. Player 2 considers routes

$$\begin{aligned} r^{(2)}_{1} = \{b,d,c\} \text{ and } r^{(2)}_{2} = \{b,a,c\} \end{aligned}$$

for its arrivals with rate $\lambda ^{(2)}$. Let $\Lambda =\lambda ^{(1)}+\lambda ^{(2)}$. With slight abuse of notation, let $f^{(i)}(r^{(1)}_{j_1},r^{(2)}_{j_2})$ denote player i’s payoff function if player k selects route $j_k$, $j_k=1,2$, and $B^{(i)}(r^{(k)}_{j_k})$ player i’s best response to route $j_k$, $j_k=1,2$, of player $k\ne i$.

From

$$\begin{aligned} f^{(1)}(r^{(1)}_{1},r^{(2)}_{1})&= \frac{1}{\mu _a-\lambda ^{(1)}} + \frac{1}{\mu _b-\Lambda } + \frac{1}{\mu _d-\Lambda } , \\ f^{(1)}(r^{(1)}_{2},r^{(2)}_{1})&= \frac{1}{\mu _a-\lambda ^{(1)}} + \frac{1}{\mu _c-\Lambda } + \frac{1}{\mu _d-\Lambda } \end{aligned}$$

we find that player 1’s best response to route $r^{(2)}_{1}$ is

$$\begin{aligned} B^{(1)}(r^{(2)}_{1}) = \left\{ \begin{array}{ll} r^{(1)}_{1}, &{}\quad \text{ if } \mu _b\ge \mu _c,\\ r^{(1)}_{2}, &{}\quad \text{ if } \mu _c\ge \mu _b. \end{array}\right. \end{aligned}$$

Similarly, it follows that $B^{(1)}(r^{(2)}_{2})=B^{(1)}(r^{(2)}_{1})$. Player 2’s best responses are

$$\begin{aligned} B^{(2)}(r^{(1)}_{1}) = B^{(2)}(r^{(1)}_{2})= \left\{ \begin{array}{ll} r^{(2)}_{1}, &{}\quad \text{ if } \mu _d\ge \mu _a,\\ r^{(2)}_{2}, &{}\quad \text{ if } \mu _a\ge \mu _d. \end{array}\right. \end{aligned}$$

These results also show that for $\mu _b\ge \mu _c$ route $r^{(1)}_{1}$ is a dominant strategy for player 1, and else route $r^{(1)}_{2}$ is a dominant strategy. Route $r^{(2)}_{1}$ is player 2’s dominant strategy if $\mu _d\ge \mu _a$, else route $r^{(2)}_{2}$ is a dominant strategy.

Second, assume the players share their source and sink nodes as depicted in Fig. 2b. Player j’s routes are $r^{(j)}_{1}=\{a,b,d\}$ and $r^{(j)}_{2}=\{a,c,d\}$.

From

$$\begin{aligned} f^{(1)}(r^{(1)}_{1},r^{(2)}_{1})&= \frac{1}{\mu _a-\Lambda } + \frac{1}{\mu _b-\Lambda } + \frac{1}{\mu _d-\Lambda } , \\ f^{(1)}(r^{(1)}_{2},r^{(2)}_{1})&= \frac{1}{\mu _a-\Lambda } + \frac{1}{\mu _c-\lambda ^{(1)}} + \frac{1}{\mu _d-\Lambda } \end{aligned}$$

we find that player 1’s best response to route $r^{(2)}_{1}$ is

$$\begin{aligned} B^{(1)}(r^{(2)}_{1}) = \left\{ \begin{array}{ll} r^{(1)}_{1}, &{}\quad \text{ if } \mu _b-\lambda ^{(2)}\ge \mu _c,\\ r^{(1)}_{2}, &{}\quad \text{ if } \mu _b-\lambda ^{(2)}<\mu _c. \end{array}\right. \end{aligned}$$

We find the best responses

$$\begin{aligned} B^{(j)}(r^{(3-j)}_{1})&= \left\{ \begin{array}{ll} r^{(j)}_{1}, &{}\quad \text{ if } \mu _b-\lambda ^{(3-j)}\ge \mu _c,\\ r^{(j)}_{2}, &{}\quad \text{ if } \mu _b-\lambda ^{(3-j)}\le \mu _c. \end{array}\right. \\ B^{(j)}(r^{(3-j)}_{2})&= \left\{ \begin{array}{ll} r^{(j)}_{1}, &{}\quad \text{ if } \mu _b+\lambda ^{(3-j)}\ge \mu _c,\\ r^{(j)}_{2}, &{}\quad \text{ if } \mu _b+\lambda ^{(3-j)}\le \mu _c. \end{array}\right. \end{aligned}$$

for player $j=1,2$. For ease of exposition, assume that $\lambda ^{(1)}\ge \lambda ^{(2)}$. We distinguish five cases:

Case I: $\mu _c\le \mu _b-\lambda ^{(1)}$. In this case route $r^{(j)}_{1}=\{a,b,d\}$ is a dominant strategy for each player. Thus, $(r^{(1)}_{1},r^{(2)}_{1})$ is a pure-strategy Nash equilibrium.

Case II: $\mu _b-\lambda ^{(1)}\le \mu _c \le \mu _b-\lambda ^{(2)}$. Route $r^{(1)}_{1}=\{a,b,d\}$ is a dominant strategy for player 1. This implies that in this case $(r^{(1)}_{1},r^{(2)}_{2})$ is a pure-strategy Nash equilibrium.

Case III: $\mu _b-\lambda ^{(2)}\le \mu _c \le \mu _b+\lambda ^{(2)}$. From the best-response correspondences two pure-strategy Nash equilibria emerge: $(r^{(1)}_{1},r^{(2)}_{2})$ and $(r^{(1)}_{2},r^{(2)}_{1})$.

Case IV: $\mu _b+\lambda ^{(2)}\le \mu _c \le \mu _b+\lambda ^{(1)}$. Route $r^{(1)}_{2}=\{a,c,d\}$ is a dominant strategy for player 1. This implies that in this case $(r^{(1)}_{2},r^{(2)}_{1})$ is a pure-strategy Nash equilibrium.

Case V: $\mu _b+\lambda ^{(1)}\le \mu _c$. Now route $r^{(j)}_{2}=\{a,c,d\}$ is a dominant strategy for each player j. This implies that in this case $(r^{(1)}_{2},r^{(2)}_{2})$ is a pure-strategy Nash equilibrium.

We conclude that there exists a pure-strategy Nash equilibrium for 2-player games on a $2\times 2$-grid. $\square$

We now return to the key questions on route preferences for the cases in Fig. 2, as introduced prior to Theorem 5. In case (a), where a player has to share the opponent’s source or sink node and the question is which one will be selected, independent of the opponent’s selection, a player will select the node with the larger service rate. In case (b), where a player may share the intermediate node with the opponent and the question is if the player wants to do so or not, the best reply depends on the opponent’s route choice. Given the opponent’s route choice, a player compares the service rate of the free intermediate node to the ’remaining’ service capacity (the difference between the service rate of the node and the arrival rate of the opponent) of the intermediate node chosen by the opponent, and prefers the node with the larger remaining service capacity.

Now we consider games on a general $A\times B$-grid. If the source and sink nodes of the players are such that their routes $r^{(j)}$ for player j, $j=1,2$, intersect in at least one node, i.e., $r^{(1)}\cap r^{(2)}\notin \emptyset$, then a sufficient additional condition for feasibility of a strategy profile p is $\lambda ^{(1)}+\lambda ^{(2)}<\mu _i$ for all nodes $i\in r^{(1)}\cap r^{(2)}$. The mean sojourn times of the customers of the players are

$$\begin{aligned} f^{(1)}(p)= & {} \sum _{\{i \in r^{(1)}\setminus r^{(2)}\}} \frac{1}{\mu _i-\lambda ^{(1)}} +\sum _{\{i \in r^{(1)}\cap r^{(2)}\}}\frac{1}{\mu _i-(\lambda ^{(1)}+\lambda ^{(2)})}, \\ f^{(2)}(p)= & {} \sum _{\{i \in r^{(2)}\setminus r^{(1)}\}} \frac{1}{\mu _i-\lambda ^{(2)}} +\sum _{\{i \in r^{(1)}\cap r^{(2)}\}}\frac{1}{\mu _i-(\lambda ^{(1)}+\lambda ^{(2)})} .\\ \end{aligned}$$

For the case $\mu _i=\mu$, $i \in {\mathcal {C}}$, if $2\lambda <\mu$ the game has a pure-strategy Nash equilibrium according to Theorem 4. For this Nash equilibrium the intersection of the routes will be in a single node: $r^{(1)}\cap r^{(2)}=\{j\}$, since sharing multiple nodes clearly increases the mean sojourn time of the customers of both players. The following theorem shows that the game has a pure-strategy Nash equilibrium when the service rates $\mu _i$ are confined to two values. Without loss of generality, we may assume that $\mu _i \in \{\mu ,k\mu \}$, for $k > 1$, $i \in {\mathcal {C}}$, and that $\lambda ^{(1)}=m\lambda$, $\lambda ^{(2)}=\lambda$, for $m<1$.

Theorem 6

The 2-player game on a network of single server queues on a grid with $\mu _i \in \{\mu ,k\mu \}$, for $k > 1$, $i \in {\mathcal {C}}$, and $\lambda ^{(1)}=m\lambda$, $\lambda ^{(2)}=\lambda$, $m<1$, such that

$$\begin{aligned} \frac{k-1}{m}< \frac{\lambda }{\mu } <\frac{1}{m+1} \end{aligned}$$

(20)

has a pure-strategy Nash equilibrium.

The condition $\frac{\lambda }{\mu } <\frac{1}{m+1}$ allows two routes to share a slow node with rate $\mu$. The condition $\frac{k-1}{m} < \frac{\lambda }{\mu }$ will guarantee that a route that is a best response will intersect in at most 1 node with the other route, as it implies that

$$\begin{aligned} \frac{1}{k\mu -(m+1)\lambda } > \frac{1}{\mu -\lambda } , \end{aligned}$$

so also in a fast node with rate $k\mu$ intersection of two routes will incur a larger sojourn time than that of a single route in a slow node. Note that (20) implies that $k<\frac{2m+1}{m+1}$. As $m<1$ this implies that $1<k<\frac{3}{2}$ so that the faster server may be at most 50$\%$ faster than the slower server.

Proof

Let player 1 select a route $r^{(1)}$ that minimizes the sojourn time of its customers, ignoring player 2’s customers, and let player 2 select a route $r^{(2)}$ that is a best response to the route of player 1. If these routes do not intersect, then the strategy profile selecting these routes is a pure-strategy Nash equilibrium.

Now assume these routes intersect in node $i\in r^{(1)}\cap r^{(2)}$. There are two possible values for the service rates: Case I: $\mu _i=\mu$ and Case II: $\mu _i=k\mu$. For each case we will consider player 1’s best response $r'^{(1)}$ to the route of player 2, which will also intersect with $r^{(2)}$ in one node. Let that be node $i'\in r'^{(1)}\cap r^{(2)}$.

Case I: $\mu _i = \mu$. If $i'=i$, then the original route profile $(r^{(1)},r^{(2)})$ is a pure-strategy Nash equilibrium. If $i'\ne i$, first assume that $\mu _{i'}=\mu$. Note that player 2’s payoff is unaffected. Further note that the original route $r^{(1)}$ selected by player 1 was optimal ignoring player 2’s customers, and that player 1’s payoff at the intersection of the routes does not change if the intersecting node is $i'$ instead of i since the service rates of these nodes coincide. Therefore, it must be that the payoff at the non-intersecting part of player 1’s route does not change. Hence, the original route profile $(r^{(1)},r^{(2)})$ is a pure-strategy Nash equilibrium. Second, assume that $\mu _{i'}=k\mu$. Then, we have two routes intersecting in a fast node, i.e., Case II. Below, we show that the new routes $(r^{(1)},r^{(2)})$ yield a pure-strategy Nash equilibrium.

Case II: $\mu _i = k\mu$. We will show that under the conditions of the theorem these routes yield a pure-strategy Nash equilibrium. If $i'=i$, then the original route profile $(r^{(1)},r^{(2)})$ is a pure-strategy Nash equilibrium. If $i'\ne i$, then first assume that $\mu _{i'}=k\mu$. This will not affect the payoff of player 2. Following similar arguments as for case I, the part of the route of player 1 that does not intersect with the route of player 2 cannot have a smaller sojourn time. Hence, the route profile $(r^{(1)},r^{(2)})$ is a pure-strategy Nash equilibrium. Second, assume $\mu _{i'}=\mu$. For player 1, the new route $r'^{(1)}$ only results in a reduced sojourn time if the other nodes on route $r'^{(1)}$ compensate for the increased sojourn time in node $i'$. The original route $r^{(1)}$ of player 1 was optimal. Therefore, the new route for player 1, which intersects in node $i'$ with service rate $\mu _{i'}=\mu$, cannot contain more nodes with high service rate $k\mu$ than the original route. If the new route $r'^{(1)}$ has one node with low service rate more than the original route, than the sojourn time of player 1 should have increased. This contradicts our assumption of $r'^{(1)}$ being a best response. If the new route has the same number of nodes with high service rate, and therefore $r'^{(1)}\setminus r^{(2)}$ contains one extra high service rate node, the difference in sojourn times $\Delta f^{(1)}:=f^{(1)}(r'^{(1)},r^{(2)})- f^{(1)}(r^{(1)},r^{(2)})$ for player 1 on routes $r'^{(1)}$ and $r^{(1)}$ is

$$\begin{aligned} \Delta f^{(1)}=\left( \frac{1}{\mu -(m+1)\lambda } + \frac{1}{k\mu -m\lambda }\right) - \left( \frac{1}{k\mu -(m+1)\lambda } +\frac{1}{\mu -m\lambda }\right) , \end{aligned}$$

(21)

which can readily be seen to be positive for $k>1$, and $(m+1)\lambda <\mu$. This contradicts the assumption that the new route is a best response to player 2’s route. Hence, the original route profile $(r^{(1)},r^{(2)})$ is a pure-strategy Nash equilibrium. $\square$

Observe that we may also interpret (21) as the difference between adding player 2’s customers to a fast node on the route of player 1 that increases player 1’s sojourn time by

$$\begin{aligned} \frac{1}{k\mu -(m+1)\lambda } - \frac{1}{k\mu -m\lambda } = \frac{\lambda }{({k\mu -(m+1)\lambda })(k\mu -m\lambda )} \end{aligned}$$

(22)

and adding player 2’s customers to a slow node on the route of player 1 that increases player 1’s sojourn time by

$$\begin{aligned} \frac{1}{\mu -(m+1)\lambda } - \frac{1}{\mu -m\lambda } = \frac{\lambda }{({\mu -(m+1)\lambda })({\mu -m\lambda })}. \end{aligned}$$

(23)

Clearly, for $k>1$, and $(m+1)\lambda <\mu$, the quantity (23) is larger than (22), implying that the increase in sojourn time would be larger from sharing a slow node than from sharing a fast node.

We are not able to demonstrate the existence of a pure-strategy Nash equilibrium in general. We have carried out numerical experiments to investigate existence of a pure-strategy Nash equilibrium in a randomly generated set of networks. To this end, we have performed over a million experiments on general N-player games on networks for different values of N. The games are constructed as follows. A random amount of nodes is distributed uniformly over the plane. Then, for each player, a number of routes are selected, where each route consists of a random subset of the nodes. In addition, the service rates are randomly selected, but high enough to ensure that all strategies are feasible, i.e., with service rate exceeding the sum of all arrival rates of the players that may use that node. We have observed that a pure-strategy Nash equilibrium exists in most of the random networks we thus constructed.

Some insight in the structure of the pure-strategy Nash equilibrium is obtained by considering the N-player game on a network of single server queues on a grid with $\mu _i=\mu$ for all i and $\lambda ^{(j)}=\lambda$ for all j. For each pair of players, either their routes must intersect in at least one node, or the players may select routes that do not intersect. A strategy p that attains the minimal number of intersections required in the routes of all players is a pure-strategy Nash equilibrium, since deviating from this strategy either does not increase the number of intersection and thus does not influence the mean sojourn time, or increases the number of intersections and therefore increases the mean sojourn time by at least $\frac{1}{\mu -2 \lambda }-\frac{1}{\mu -\lambda }$. The pure-strategy Nash equilibrium also minimizes the total mean sojourn time for all players.

5 Conclusions

In this paper, we have considered a new type of games: non-cooperative games on a network of single server queues, where multiple players select routes through a network to minimize the sojourn time of their customers.

In case of continuous strategy spaces, each player is allowed to distribute its customers over multiple fixed routes. We have shown that a pure-strategy Nash equilibrium exists that can be found with a best-response algorithm. In case of discrete strategy spaces, each player is allowed to select a single route for all its customers. The game has a Nash equilibrium in mixed strategies. We have illustrated via an example that such games need not have a pure-strategy Nash equilibrium. We have shown the existence of pure-strategy Nash equilibria for four subclasses of games on a network of single server queues: (i) N-player games with equal arrival rates for the players, (ii) 2-player games with identical service rates for all nodes, (iii) 2-player games on a $2\times 2$-grid, and (iv) 2-player games on an $A\times B$-grid with small differences in the service rates.

A challenging topic is comparison of the optimal strategies in the different cases. It is clear that given the feasible strategies chosen by the other players, in a pure-strategy Nash equilibrium the minimal sojourn time for the customers of a single player in the continuous case is lower than in the discrete case as the mathematical program (7)–(11) in the continuous case is a relaxation of the discrete case (2)–(6). However, as shown in the first continuation of Example 1 for a single player in a pure-strategy Nash equilibrium the minimal sojourn time in the discrete case might be lower than in the continuous case. This is because the other player in the continuous case will select a strategy that minimizes its own sojourn time, and this strategy is not feasible in the discrete case. In addition, the mixed strategies in the discrete case do not coincide with the pure strategies for the continuous case, as illustrated in the second continuation of Example 1. These differences are due to the nature of the Nash equilibrium that is an equilibrium in which no player can unilaterally change its strategy to decrease its expected sojourn time. The Nash equilibrium does not guarantee minimal sojourn times. An interesting question for further research is into network topologies for which the Nash equilibrium also gives the minimal expected sojourn times for all players.

In Example 2 a specific 6-node network was used to construct a game without a pure-strategy Nash equilibrium. This gives rise to a new research question: for which network topologies do pure-strategy Nash equilibria always exist, regardless of the number of players, consumers’ arrival rates and nodes’ service rates? This question is related to the problem of topological existence for weighted network congestion games studied in [15]. Those results are not applicable here since the costs are incurred at the network’s edges instead of its nodes and the cost functions differ. Therefore, existence of pure-strategy Nash equilibria remains a challenging topic for further research.