Handbook of Dynamic Game Theory pp 135  Cite as
Stackelberg Routing on Parallel Transportation Networks
 4 Citations
 327 Downloads
Abstract
This chapter presents a game theoretic framework for studying Stackelberg routing games on parallel transportation networks. A new class of latency functions is introduced to model congestion due to the formation of physical queues, inspired from the fundamental diagram of traffic. For this new class, some results from the classical congestion games literature (in which latency is assumed to be a nondecreasing function of the flow) do not hold. A characterization of Nash equilibria is given, and it is shown, in particular, that there may exist multiple equilibria that have different total costs. A simple polynomialtime algorithm is provided, for computing the best Nash equilibrium, i.e., the one which achieves minimal total cost. In the Stackelberg routing game, a central authority (leader) is assumed to have control over a fraction of the flow on the network (compliant flow), and the remaining flow responds selfishly. The leader seeks to route the compliant flow in order to minimize the total cost. A simple Stackelberg strategy, the noncompliant first (NCF) strategy, is introduced, which can be computed in polynomial time, and it is shown to be optimal for this new class of latency on parallel networks. This work is applied to modeling and simulating congestion mitigation on transportation networks, in which a coordinator (traffic management agency) can choose to route a fraction of compliant drivers, while the rest of the drivers choose their routes selfishly.
Keywords
Transportation networks Nonatomic routing game Stackelberg routing game Nash equilibrium Fundamental diagram of traffic Price of stability1 Introduction
1.1 Motivation and Related Work
Routing games model the interaction between players on a network, where the cost for each player on an edge depends on the total congestion of that edge. Extensive work has been dedicated to the study of Nash equilibria for routing games (or Wardrop equilibria in the transportation literature, Wardrop 1952), in which players selfishly choose the routes that minimize their individual costs (latencies) (Beckmann et al. 1956; Dafermos 1980; Dafermos and Sparrow 1969). In general, Nash equilibria are inefficient compared to a system optimal assignment that minimizes the total cost on the network (Koutsoupias and Papadimitriou 1999). This inefficiency has been characterized for different classes of latency functions and network topologies (Roughgarden and Tardos 2004; Swamy 2007). This helps understand the inefficiencies caused by congestion in communication networks and transportation networks. In order to reduce the inefficiencies due to selfish routing, many instruments have been studied, including congestion pricing (Farokhi and Johansson 2015; Ozdaglar and Srikant 2007), capacity allocation (Korilis et al. 1997b), and Stackelberg routing (Aswani and Tomlin 2011; Korilis et al. 1997a; Roughgarden 2001; Swamy 2007).
1.1.1 Online Learning and Decision Dynamics in the Routing Game
The Nash equilibrium concept gives a characterization of the state of a network at equilibrium but does not specify how players arrive to the equilibrium. The study of decision dynamics provides an answer to this question and has been a fundamental topic in economics (Blume 1993), game theory (Shamma 2015; Weibull 1997), and online learning theory (CesaBianchi and Lugosi 2006). These models usually assume that the game is played repeatedly (as opposed to a oneshot game) and that each player faces a sequential decision problem: At each iteration, the player takes an action and observes an outcome (which is also affected by the decisions of other players). The player can then use the outcome to update her decision on the next iteration. One of the natural questions that can be studied is whether the joint player decisions converge to an invariant set (typically, the Nash equilibrium of the oneshot game, or some other equilibrium concept). This question has a long history that dates back to Hannan (1957) who defined the regret and Blackwell (1956) who defined approachability, which became essential tools in the modeling and analysis of repeated games and convergence of player dynamics.
Decision dynamics have since been studied for several classes of games, such as potential games (Monderer and Shapley 1996), and many results provide convergence guarantees under different classes of decision dynamics (Benaïm 2015; Fox and Shamma 2013; Hofbauer and Sandholm 2009; Sandholm 2001). Although we do not study decision dynamics in this chapter, we review some of the work most relevant to routing games.
Routing games are a special case of potential games (Sandholm 2010), and decisions dynamics have been studied in the context of routing games: Blum et al. (2006) study general noregret dynamics, Kleinberg et al. (2009) and Krichene et al. (2015a,b) study other classes of dynamics for which they give stronger convergence guarantees, and Fischer et al. (2010) studies a similar, samplingbased model. Several of these results relate the discrete algorithm to a continuoustime limit known as the replicator ODE, which is well studied in evolutionary game theory in general (Weibull 1997), and in routing games in particular (Drighès et al. 2014; Fischer and Vöcking 2004). Several studies build on these models of decision dynamics, to pose and solve estimation and control problems, such as estimating the latency functions on the network (Thai et al. 2015), estimating the learning rates of the dynamics (Lam et al. 2016), and solving optimal routing under selfish response (Krichene et al. 2016).
1.1.2 Stackelberg Routing Games
In the Stackelberg routing game, a subset of the players, corresponding to a fraction of the total flow, hereafter called the compliant flow, is centrally assigned by a coordinator (leader), then the remaining players (followers) choose their routes selfishly. The objective of the leader is to assign the compliant flow in a manner that minimizes a systemwide cost function, while anticipating the followers’ selfish response. This setting is relevant in the planning and operation of transportation and communication networks. In transportation networks, advances in traveler information systems have made it possible to interact with individual drivers and exchange information through GPSenabled smartphone applications or vehicular navigation systems (Work et al. 2010). These devices can be used by a traffic control center to provide routing advice that can improve the overall efficiency of the network. Naturally, the question arises on how the traffic control center should coordinate with the compliant drivers while accounting for the selfish response of other drivers, hence the importance of the Stackelberg routing framework. One might argue that the drivers who are offered routing advice are not guaranteed to follow the suggested routes, especially when these routes do not have minimal latency (in order to improve the systemwide efficiency, some drivers will be assigned routes that are suboptimal in the Nash sense). However, in some cases, it can be reasonably assumed that a fraction of the drivers will choose the routes suggested by the coordinator, despite immediate fairness concerns. For example, some drivers may have sufficient external incentives to be compliant with the coordinator. In addition, the compliant flow may also include drivers who care about the systemwide efficiency.
Stackelberg routing on parallel networks has been studied for the class of nondecreasing latency functions, and it is known that computing the optimal Stackelberg strategy is NPhard (Roughgarden 2001). This led to the design of polynomial time approximate strategies such as largest latency first (Roughgarden 2001; Swamy 2007). While this class of latency functions provides a good model of congestion for a broad range of networks such as communication networks, it does not fully capture congestion phenomena in transportation. The main difference is that in transportation networks, the queuing of traffic results in an increase in density of vehicles (Daganzo 1994; Lebacque 1996; Lighthill and Whitham 1955; Richards 1956; Work et al. 2010), which in turn affects the latency. This phenomenon is sometimes referred to as horizontal queueing, since the queuing of vehicles takes physical space, as opposed to vertical queuing, such as queuing of packets in a communication link, which does not take physical space, and the notion of density is absent. Several authors have proposed different models of congestion to capture congestion phenomena specific to horizontal queuing and characterized the Nash equilibria under these models (Boulogne et al. 2001; Friesz and Mookherjee 2006; Lo and Szeto 2002; Wang et al. 2001). We introduce a new class of latency functions for congestion with horizontal queuing and study Nash and Stackelberg equilibria under this class. We restrict our study to parallel networks. Although simple, the parallel topology can be of practical importance in several situations, such as traffic planning and analysis. Even though transportation networks are rarely parallel, they can be approximated by a parallel network, for example, by only considering highways that connect two highly populated areas (Caltrans 2010). Figure 9 shows one such network that connects San Francisco to San Jose. We consider this network in Sect. 6.
1.2 Congestion on Horizontal Queues

There exists a unique density ρ_{ n }^{crit} ∈ (0, ρ_{ n }^{max}) such that x_{ n }^{ ρ }(ρ_{ n }^{crit}) = x_{ n }^{max}, called critical density. When ρ_{ n } ∈ [0, ρ_{ n }^{crit}], the link is said to be in freeflow, and when ρ_{ n } ∈ (ρ_{ n }^{crit}, ρ_{ n }^{max}), it is said to be congested.

In the congested regime, x_{ n }^{ ρ } is continuous decreasing from (ρ_{ n }^{crit}, ρ_{ n }^{max}) onto (0, x_{ n }^{max}). In particular, \(\lim _{\rho _{n}\rightarrow \rho _{n}^{\max }}x_{n}^{\rho }(\rho _{n}) = 0\) (the flow reduces to zero when the density approaches the maximum density).

Either a large concentration of cars moving slowly (high density, the road is congested), in which case the latency is large

Or few cars moving fast (low density, the road is in freeflow), in which case the latency is small
1.3 Latency Function for Horizontal Queues

In the freeflow regime, the flux function is linearly increasing, \(x_{n}(\rho _{n}) = \frac{x_{n}^{\max }} {\rho _{n}^{\text{crit}}}\rho _{n}\). Thus the latency is constant in freeflow, \(\ell_{n}^{\rho }(\rho _{n}) = \frac{L_{n}\rho _{n}^{\text{crit}}} {x_{n}^{\max }}\). We will denote its value by \(a_{n}\mathop{ =}\limits^{ \Delta }\frac{L_{n}\rho _{n}^{\text{crit}}} {x_{n}^{\max }}\), called henceforth the freeflow latency.
 In the congested regime, x_{ n }^{ ρ } is bijective from (ρ_{ n }^{crit}, ρ_{ n }^{max}) to (0, x_{ n }^{max}). Letbe its inverse. It maps the flow x_{ n } to the unique congestion density that corresponds to that flow. Thus in the congested regime, latency can be expressed as a function of the flow, x_{ n } ↦ ℓ_{ n }^{ ρ }(ρ_{ n }^{cong}(x_{ n })). This function is decreasing as the composition of the decreasing function ρ_{ n }^{cong} and the increasing function ℓ_{ n }^{ ρ }.$$\displaystyle{\begin{array}{rl} \rho _{n}^{\text{cong}}: (0,x_{n}^{\max })& \rightarrow (\rho _{n}^{\text{crit}},\rho _{n}^{\max }) \\ x_{n}&\mapsto \rho _{n}^{\text{cong}}(x_{n})\end{array} }$$
We can therefore express the latency as a function of the flow in each of the separate regimes: freeflow (low density) and congested (high density). This leads to the following definition of HQSF latencies (horizontal queues, singlevalued in freeflow). We introduce a binary variable m_{ n } ∈ {0, 1} which specifies whether the link is in the freeflow or the congested regime.
Definition 1 (HQSF latency class)
 (A1)
In the freeflow regime, the latency ℓ_{ n }(⋅ , 0) is single valued (i.e., constant).
 (A2)
In the congested regime, the latency x_{ n } ↦ ℓ_{ n }(x_{ n }, 1) is decreasing on (0, x_{ n }^{max}).
 (A3)
\(\lim _{x_{n}\rightarrow x_{n}^{\max }}\ell_{n}(x_{n},1) = a_{n} =\ell _{n}(x_{n}^{\max },0)\).
Property (A1) is equivalent to the assumption that the flux function is linear in freeflow. Property (A2) results from the expression of the latency as the composition ℓ_{ n }^{ ρ }(ρ_{ n }^{cong}(x_{ n })), where ℓ_{ n }^{ ρ } is increasing and ρ_{ n }^{cong} is decreasing. Property (A3) is equivalent to the continuity of the underlying flux function x_{ n }^{ ρ }.
Although it may be more natural to think of the latency as a nondecreasing function of the density, the above representation in terms of flow x_{ n } and congestion state m_{ n } will be useful in deriving properties of the Nash equilibria of the routing game. Finally, we observe, as an immediate consequence of these properties, that the latency in congestion is always greater than the freeflow latency: \(\forall x_{n} \in (0,x_{n}^{\max })\), ℓ_{ n }(x_{ n }, 1) > a_{ n }. Some examples of HQSF latency functions (and the underlying flux functions) are illustrated in Fig. 1. We now give a more detailed derivation of a latency function from a macroscopic fundamental diagram of traffic.
1.4 A HQSF Latency Function from a Triangular Fundamental Diagram of Traffic
This defines a function ℓ_{ n } that satisfies the assumptions of Definition 1 and thus belongs to the HQSF latency class. Figure 1 shows one example of a triangular fundamental diagram (top left) and the corresponding latency function ℓ_{ n } (top right).
2 Game Model and Main Results
2.1 The Routing Game
The network has a single source and a single sink. Connecting the source and sink are N parallel links indexed by \(n \in \left \{1,\ldots,N\right \}\). We assume, without loss of generality, that the links are ordered by increasing freeflow latencies. To simplify the discussion, we further assume that freeflow latencies are distinct. Therefore we have \(a_{1} <a_{2} <\ldots <a_{N}\). The network is subject to a constant positive flow demand r at the source. We will denote by (N, r) an instance of the routing game played on a network with N parallel links subject to demand r. The state of the network is given by a feasible flow assignment vector \(\boldsymbol{x} \in \mathbb{R}_{+}^{N}\) such that ∑_{n = 1}^{ N }x_{ n } = r where x_{ n } is the flow on link n and a congestion state vector \(\boldsymbol{m} \in \{ 0,1\}^{N}\) where m_{ n } = 0 if the link is in freeflow and m_{ n } = 1 if the link is congested, as defined above. All physical quantities (density and flow) are assumed to be static and uniform on the link.
Every nonatomic player chooses a route in order to minimize his/her individual latency (Roughgarden and Tardos 2002). If a player chooses link n, his/her latency is given by ℓ_{ n }(x_{ n }, m_{ n }), where ℓ_{ n } is a HQSF latency function. We assume that players know the latency functions.
Pure Nash equilibria of the game (which we will simply refer to as Nash equilibria) are assignments \((\boldsymbol{x},\boldsymbol{m})\) such that every player cannot improve his/her latency by switching to a different link.
Definition 2 (Nash Equilibrium)
A feasible assignment \((\boldsymbol{x},\boldsymbol{m}) \in \mathbb{R}_{+}^{N} \times \{ 0,1\}^{N}\) is a Nash equilibrium of the routing game instance (N, r) if \(\forall n \in \text{supp}\left (\boldsymbol{x}\right )\), \(\forall k \in \{ 1,\ldots,N\}\), ℓ_{ n }(x_{ n }, m_{ n }) ≤ ℓ_{ k }(x_{ k }, m_{ k }).
While a Nash equilibrium achieves minimal individual latencies, it does not minimize, in general, the system cost or total cost defined as follows:
Definition 3
As detailed in Sect. 3, under the HQSF latency class, the routing game may have multiple Nash equilibria that have different total costs. We are interested, in particular, in Nash equilibria that have minimal cost, which are referred to as best Nash equilibria (BNE).
Definition 4 (Best Nash Equilibria)
2.2 Stackelberg Routing Game

First, the coordinator (the leader) chooses a Stackelberg strategy, i.e., an assignment \(\boldsymbol{s} \in \mathbb{R}_{+}^{N}\) of the compliant flow (such that ∑_{n = 1}^{ N }s_{ n } = αr).
 Then, the Stackelberg strategy \(\boldsymbol{s}\) of the leader is revealed, and the noncompliant players (followers) choose their routes selfishly and form a Nash equilibrium \((\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\), induced^{2} by strategy \(\boldsymbol{s}\). By definition, the induced equilibrium \((\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\) satisfies$$\displaystyle\begin{array}{rcl} & & \forall n \in \text{supp}\left (\boldsymbol{t}(\boldsymbol{s})\right )\!,\ \forall k \in \{ 1,\ldots,N\}, \\ & & \qquad \qquad \qquad \quad \ \ \qquad \ell_{n}(s_{n} + t_{n}(\boldsymbol{s}),m_{n}(\boldsymbol{s})) \leq \ell_{k}(s_{k} + t_{k}(\boldsymbol{s}),m_{k}(\boldsymbol{s})) {}\end{array}$$(5)
The total flow on the network is \(\boldsymbol{s} +\boldsymbol{ t}(\boldsymbol{s})\); thus the total cost is \(C(\boldsymbol{s} +\boldsymbol{ t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\). Note that a Stackelberg strategy \(\boldsymbol{s}\) may induce multiple Nash equilibria in general. However, we define \((\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\) to be the best such equilibrium (the one with minimal total cost, which will be shown to be unique in Sect. 4).

(N, r, α) is an instance of the Stackelberg routing game played on a parallel network with N links under flow demand r with compliance rate α. Note that the routing game (N, r) is a special case of the Stackelberg routing game with α = 0.

\(\text{S}(N,r,\alpha ) \subset \mathbb{R}_{+}^{N}\) is the set of Stackelberg strategies for the Stackelberg instance (N, r, α).
 S^{⋆}(N, r, α) is the set of optimal Stackelberg strategies defined as$$\displaystyle{ \text{S}^{\star }(N,r,\alpha ) =\mathop{\arg \min }\limits_{\boldsymbol{ s} \in \text{S}(N,r,\alpha )}C(\boldsymbol{s} +\boldsymbol{ t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s})). }$$(6)
2.3 Optimal Stackelberg Strategy
We now define a candidate Stackelberg strategy, which we call the noncompliant first strategy (NCF) and which we will prove to be optimal. The NCF strategy corresponds to first computing the best Nash equilibrium \((\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\) of the noncompliant flow for the routing game instance \(\big(N,(1\alpha )r\big)\), then finding a particular strategy \(\boldsymbol{s}\) that induces \((\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\).
Definition 5 (The noncompliant first (NCF) strategy)
Consider the Stackelberg instance (N, r, α). Let \((\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\) be the best Nash equilibrium of the noncompliant flow, \(\{(\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\} = \text{BNE}(N,(1\alpha )r)\), and \(\bar{k} =\max \text{supp}(\bar{\boldsymbol{t}})\) be the last link in its support. Then the noncompliant first strategy, denoted by NCF(N, r, α), is defined as follows:
where l is the maximal index in \(\{\bar{k} + 1,\ldots,N\}\) such that \(\alpha r\! \!\big (\!\sum _{n=\bar{k}}^{l1}x_{n}^{\max }\! \!\bar{ t}_{\bar{k}}\!\big) \geq 0\).
Theorem 1
Under the class of HQSF latency functions, NCF(N, r, α) is an optimal Stackelberg strategy for the Stackelberg instance (N, r, α).
We give a proof of Theorem 1 in Sect. 4. We will also show that for the class of HQSF latency functions, the best Nash equilibria can be computed in polynomial time in the size N of the network, and as a consequence, the NCF strategy can also be computed in polynomial time. This stands in contrast to previous results under the class of nondecreasing latency functions, for which computing the optimal Stackelberg strategy is NPhard (Roughgarden 2001).
3 Nash Equilibria
In this section, we study Nash equilibria of the routing game. We show that under the class of HQSF latency functions, there may exist multiple Nash equilibria that have different costs. Then we partition the set of equilibria into congested equilibria and singlelinkfreeflow equilibria. Finally, we characterize the best Nash equilibrium and show that it can be computed in quadratic time in the number of links.
3.1 Structure and Properties of Nash Equilibria
We first give some properties of Nash equilibria.
Proposition 1 (Total cost of a Nash Equilibrium)
Let \((\boldsymbol{x},\boldsymbol{m}) \in {\text{NE}}(N,r)\) be a Nash equilibrium for the instance (N, r). Then there exists ℓ_{0} > 0 such that \(\forall n \in \mathit{\text{supp}}\left (\boldsymbol{x}\right )\), ℓ_{ n }(x_{ n }, m_{ n }) = ℓ_{0}, and \(\forall n\notin \mathit{\text{supp}}\left (\boldsymbol{x}\right )\), ℓ_{ n }(0, 0) ≥ ℓ_{0}. The total cost of the equilibrium is then \(C(\boldsymbol{x},\boldsymbol{m}) = r\ell_{0}\).
Proposition 2
Let \((\boldsymbol{x},\boldsymbol{m}) \in {\text{NE}}(N,r)\) be a Nash equilibrium. Then \(k \in \mathit{\text{supp}}\left (\boldsymbol{x}\right ) \Rightarrow\) \(\forall n <k\) , link n is congested.
Proof
By contradiction, if m_{ n } = 0, then ℓ_{ n }(x_{ n }, m_{ n }) = a_{ n } < a_{ k } ≤ ℓ_{ k }(x_{ k }, m_{ k }), which contradicts Definition 2 of a Nash equilibrium.
Corollary 1 (Support of a Nash equilibrium)
Let \((\boldsymbol{x},\boldsymbol{m}) \in {\text{NE}}(N,r)\) be a Nash equilibrium and \(k =\max \mathit{\text{supp}}\left (\boldsymbol{x}\right )\) be the last link in the support of \(\boldsymbol{x}\) (i.e., the one with the largest freeflow latency). Then we have \(\mathit{\text{supp}}\left (\boldsymbol{x}\right ) =\{ 1,\ldots,k\}\).
Proof
Since \(k \in \text{supp}\left (\boldsymbol{x}\right )\), we have by Proposition 2 that \(\forall n <k\), link n is congested, thus \(n \in \text{supp}\left (\boldsymbol{x}\right )\) (by definition, a congested link cannot be empty).
3.1.1 No Essential Uniqueness
For the HQSF latency class, the essential uniqueness property^{3} does not hold, i.e., there may exist multiple Nash equilibria that have different costs; an example is given in Fig. 3.
3.1.2 SingleLinkFreeFlow Equilibria and Congested Equilibria
 Either , i.e., the last link in the support is in freeflow, all other links in the support are congested. In this case we call \((\boldsymbol{x},\boldsymbol{m})\) a singlelinkfreeflow equilibrium and denote the set of such equilibria by NE_{f}(N, r).
 Or , i.e., all links in the support are congested. In this case we call \((\boldsymbol{x},\boldsymbol{m})\) a congested equilibrium and denote the set of such equilibria by NE_{c}(N, r).
3.2 Existence of SingleLinkFreeFlow Equilibria
Let \((\boldsymbol{x},\boldsymbol{m})\) be a singlelinkfreeflow equilibrium, and let \(k =\max \text{supp}\left (\boldsymbol{x}\right )\). We have from Proposition 2 that links \(\left \{1,\ldots,k  1\right \}\) are congested and link k is in freeflow. Therefore we must have \(\forall n \in \{ 1,\ldots,k  1\}\), ℓ_{ n }(x_{ n }, 1) = ℓ_{ k }(x_{ k }, 0) = a_{ k }. This uniquely determines the flow on the congested links:
Definition 6 (Congestion flow)
Proposition 3 (Singlelinkfreeflow equilibria)
Next, we give a necessary and sufficient condition for the existence of singlelinkfreeflow equilibria.
Lemma 1
Proof
If a singlelinkfreeflow equilibrium exists, then by Proposition 3, it is of the form given by Eqs. (10) and (9) for some k. The flow on link k is then given by \(r \sum _{n=1}^{k1}\hat{x}_{n}(k) \leq x_{k}^{\max }\). Therefore \(r \leq x_{k}^{\max } +\sum _{ n=1}^{k1}\hat{x}_{n}(k) \leq r^{\text{NE}}(N)\).
We prove the converse by induction on the size N of the network. Let P_{ N } denote the property: \(\forall r \in (0,r^{\text{NE}}(N)]\), there exists a singlelinkfreeflow equilibrium for the instance (N, r).
For N = 1, it is clear that if 0 < r ≤ x_{1}^{max}, there is a singlelinkfreeflow equilibrium simply given by (x_{1}, m_{1}) = (r, 0).
Now let N ≥ 1, assume P_{ N } holds and let us show P_{N+1}. Let 0 < r ≤ r^{NE}(N + 1), and consider an instance (N + 1, r).
Case 1
If r ≤ r^{NE}(N), then by the induction hypothesis P_{ N }, there exists a singlelinkfreeflow equilibrium \((\boldsymbol{x},\boldsymbol{m})\) for the instance (N, r). Then \((\boldsymbol{x}',\boldsymbol{m}')\) defined as \(\boldsymbol{x}' = (x_{1},\ldots,x_{N},0)\) and \(\boldsymbol{m}' = (m_{1},\ldots,m_{N},0)\) is clearly a singlelinkfreeflow equilibrium for the instance (N + 1, r).
Case 2
Corollary 2
The maximum demand r such that the set of Nash equilibria NE(N, r) is nonempty is r^{NE}(N).
Proof
3.3 Number of Equilibria
Proposition 4 (An upper bound on the number of equilibria)
Consider a routing game instance (N, r). For any given \(k \in \{ 1,\ldots,N\}\), there is at most one singlelinkfreeflow equilibrium and one congested equilibrium with support \(\{1,\ldots,k\}\). As a consequence, by Corollary 1, the instance (N, r) has at most N singlelinkfreeflow equilibria and N congested equilibria.
Proof
We prove the result for singlelinkfreeflow equilibria, the proof for congested equilibria is similar. Let \(k \in \{ 1,\ldots,N\}\), and assume \((\boldsymbol{x},\boldsymbol{m})\) and \((\boldsymbol{x}',\boldsymbol{m}')\) are singlelinkfreeflow equilibria such that \(\max \text{supp}\left (\boldsymbol{x}\right ) =\max \text{supp}\left (\boldsymbol{x}'\right ) = k\). We first observe that by Corollary 1, \(\boldsymbol{x}\) and \(\boldsymbol{x}'\) have the same support \(\{1,\ldots,k\}\), and by Proposition 2, \(\boldsymbol{m} =\boldsymbol{ m}'\). Since link k is in freeflow under both equilibria, we have ℓ_{ k }(x_{ k }, m_{ k }) = ℓ_{ k }(x_{ k }′, m_{ k }′) = a_{ k }, and by Definition 2 of a Nash equilibrium, any link in the support of both equilibria has the same latency a_{ k }, i.e., \(\forall n <k\), ℓ_{ n }(x_{ n }, 1) = ℓ_{ n }(x_{ n }′, 1) = a_{ k }. Since the latency in congestion is injective, we have \(\forall n <k\), x_{ n } = x_{ n }′, therefore \(\boldsymbol{x} =\boldsymbol{ x}'\).
3.4 Best Nash Equilibrium
In order to study the inefficiency of Nash equilibria, and the improvement of performance that we can achieve using optimal Stackelberg routing, we focus our attention on best Nash equilibria and price of stability (Anshelevich et al. 2004) as a measure of their inefficiency.
Lemma 2 (Best Nash Equilibrium)
Proof
We first show that a congested equilibrium cannot be a best Nash equilibrium. Let \((\boldsymbol{x},\boldsymbol{m}) \in \text{NE}(N,r)\) be a congested equilibrium, and let \(k =\max \text{supp}\left (x\right )\). By Proposition 1, the cost of \((\boldsymbol{x},\boldsymbol{m})\) is \(C(\boldsymbol{x},\boldsymbol{m}) =\ell _{k}(x_{k},1)r> a_{k}r\). We observe that \((\boldsymbol{x},\boldsymbol{m})\) restricted to \(\{1,\ldots,k\}\) is an equilibrium for the instance (k, r); thus by Corollary 2, r ≤ r^{NE}(k), and by Lemma 1, there exists a singlelinkfreeflow equilibrium \((\boldsymbol{x}',\boldsymbol{m}')\) for (k, r), with cost \(C(\boldsymbol{x}',\boldsymbol{m}') \leq a_{k}r\). Clearly, \((\boldsymbol{x}'',\boldsymbol{m}'')\), defined as \(\boldsymbol{x}'' = (x_{1}',\ldots,x_{k}',0,\ldots,0)\) and \(\boldsymbol{m}'' = (m_{1}',\ldots,m_{k}',0,\ldots,0)\), is a singlelinkfreeflow equilibrium for the original instance (N, r), with cost \(C(\boldsymbol{x}'',\boldsymbol{m}'') = C(\boldsymbol{x}',\boldsymbol{m}') \leq a_{k}r <C(\boldsymbol{x},\boldsymbol{m})\), which proves that \((\boldsymbol{x},\boldsymbol{m})\) is not a best Nash equilibrium. Therefore best Nash equilibria are singlelinkfreeflow equilibria. And since the cost of a singlelinkfreeflow equilibrium \((\boldsymbol{x},\boldsymbol{m})\) is simply \(C(\boldsymbol{x},\boldsymbol{m}) = a_{k}r\) where \(k =\max \text{supp}\left (\boldsymbol{x}\right )\), it is clear that the smaller the support, the lower the total cost. Uniqueness follows from Proposition 4.
3.4.1 Complexity of Computing the Best Nash Equilibrium
Lemma 2 gives a simple algorithm for computing the best Nash equilibrium for any instance (N, r): simply enumerate all singlelinkfreeflow equilibria (there are at most N such equilibria by Proposition 4), and select the one with the smallest support. This is detailed in Algorithm 1.
Algorithm 1 Best Nash equilibrium
The congestion flow values \(\{\hat{x}_{n}(k),1 \leq n <k \leq N\}\) can be precomputed in O(N^{2}). There are at most N calls to freeFlowConfig, which runs in O(N) time; thus bestNE runs in O(N^{2}) time. This shows that the best Nash equilibrium can be computed in quadratic time.
4 Optimal Stackelberg Strategies
In this section, we prove our main result that the NCF strategy is an optimal Stackelberg strategy (Theorem 1). Furthermore, we show that the entire set of optimal strategies S^{⋆}(N, r, α) can be computed in a simple way from the NCF strategy.
Figure 4 shows the total flow \(\bar{\boldsymbol{x}}_{n} =\bar{\boldsymbol{ s}}_{n} +\bar{\boldsymbol{ t}}_{n}\) on each link. Under \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\), links \(\left \{1,\ldots,\bar{k}  1\right \}\) are congested and have latency \(a_{\bar{k}}\), links \(\left \{\bar{k},\ldots,l  1\right \}\) are in freeflow and at maximum capacity, and the remaining flow is assigned to link l.
4.1 Proof of Theorem 1subsection.1: The NCF Strategy Is an Optimal Stackelberg Strategy
Let \(\boldsymbol{s} \in S(N,r,\alpha )\) be any Stackelberg strategy and \((\boldsymbol{t},\boldsymbol{m}) = (\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\) be the best Nash equilibrium of the noncompliant flow, induced by \(\boldsymbol{s}\). To prove that the NCF startegy \(\bar{s}\) is optimal, we will compare the costs induced by s and \(\bar{s}\). Let \(\boldsymbol{x} =\boldsymbol{ s} +\boldsymbol{ t}(\boldsymbol{s})\) and \(\bar{\boldsymbol{x}} =\bar{\boldsymbol{ s}} +\bar{\boldsymbol{ t}}\) be the total flows induced by each strategy. To prove Theorem 1, we seek to show that \(C(\boldsymbol{x},\boldsymbol{m}) \geq C(\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\).
The proof is organized as follows: we first compare the supports of the induced equilibria (Lemma 3) and then show that links \(\{1,\ldots,l  1\}\) are more congested under \((\boldsymbol{x},\boldsymbol{m})\) than under \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\), in the following sense  they hold less flow and have greater latency (Lemma 4). Then we conclude by showing the desired inequality.
Lemma 3
Let \(k =\max \mathit{\text{supp}}\left (\boldsymbol{t}\right )\) and \(\bar{k} =\max \mathit{\text{supp}}\left (\bar{\boldsymbol{t}}\right )\). Then \(k \geq \bar{ k}\).
In other words, the last link in the support of \(\boldsymbol{t}(\boldsymbol{s})\) has higher freeflow latency than the last link in the support of \(\bar{\boldsymbol{t}}\).
Proof
We first note that \((\boldsymbol{s} +\boldsymbol{ t}(\boldsymbol{s}),\boldsymbol{m})\) restricted to \(\text{supp}\left (\boldsymbol{t}(\boldsymbol{s})\right )\) is a Nash equilibrium. Then since link k is in freeflow, we have \(\ell_{k}(s_{k} + t_{k}(\boldsymbol{s}),m_{k}) = a_{k}\), and since \(k \in \text{supp}\left (\boldsymbol{t}(\boldsymbol{s})\right )\), we have by definition that any other link has greater or equal latency. In particular, \(\forall n \in \left \{1,\ldots k  1\right \}\), \(\ell_{n}(s_{n} + t_{n}(\boldsymbol{s}),m_{n}) \geq a_{k}\), thus \(s_{n} + t_{n}(\boldsymbol{s}) \leq \hat{ x}_{n}(k)\). Therefore we have \(\sum _{n=1}^{k}s_{n} + t_{n}(\boldsymbol{s}) \leq \sum _{n=1}^{k1}\hat{x}_{n}(k) + x_{k}^{\max }\). But \(\sum _{n=1}^{k}(s_{n} + t_{n}(\boldsymbol{s})) \geq \sum _{n\in \text{supp}\left (\boldsymbol{t}\right )}t_{n}(\boldsymbol{s}) = (1\alpha )r\) since \(\text{supp}\left (\boldsymbol{t}\right ) \subseteq \left \{1,\ldots,k\right \}\). Therefore \((1\alpha )r \leq \sum _{n=1}^{k1}\hat{x}_{n}(k) + x_{k}^{\max }\). By Lemma 1, there exists a singlelinkfreeflow equilibrium for the instance (N, (1 −α)r) supported on the first k links. Let \((\tilde{\boldsymbol{t}},\tilde{\boldsymbol{m}})\) be such an equilibrium. The cost of this equilibrium is (1 −α)rℓ_{0} where ℓ_{0} ≤ a_{ k } is the freeflow latency of the last link in the support of \(\tilde{\boldsymbol{t}}\). Thus \(C(\tilde{\boldsymbol{t}},\tilde{\boldsymbol{m}}) \leq (1\alpha )ra_{k}\). Since by definition \((\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\) is the best Nash equilibrium for the instance (N, (1 −α)r) and has cost \((1\alpha )ra_{\bar{k}}\), we must have \((1\alpha )ra_{\bar{k}} \leq (1\alpha )ra_{k}\), i.e., \(a_{\bar{k}} \leq a_{k}\).
Lemma 4
Under \((\boldsymbol{x},\boldsymbol{m})\), the links \(\{1,\ldots,l  1\}\) have greater (or equal) latency and hold less (or equal) flow than under \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\), i.e., \(\forall n \in \{ 1,\ldots,l  1\}\), \(\ell_{n}(x_{n},m_{n}) \geq \ell_{n}(\bar{x}_{n},\bar{m}_{n})\) and \(x_{n} \leq \bar{ x}_{n}\).
Proof
Since \(k \in \text{supp}\left (\boldsymbol{t}\right )\), we have by definition of a Stackelberg strategy and its induced equilibrium that \(\forall n \in \{ 1,\ldots,k  1\}\), ℓ_{ n }(x_{ n }, m_{ n }) ≥ ℓ_{ k }(x_{ k }, m_{ k }) ≥ a_{ k }, see Eq. (5). We also have by definition of \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\) and the resulting latencies given by Eq. (14), \(\forall n \in \{ 1,\ldots,\bar{k}  1\}\), n is congested, and \(\ell_{n}(x_{n},m_{n}) = a_{\bar{k}}\). Thus using the fact that \(k \geq \bar{ k}\), we have \(\forall n \in \{ 1,\ldots,\bar{k}  1\}\), \(\ell_{n}(x_{n},m_{n}) \geq a_{k} \geq a_{\bar{k}} =\ell _{n}(\bar{x}_{n},\bar{m}_{n})\), and \(x_{n} \leq \hat{ x}_{n}(k) \leq \hat{ x}_{n}(\bar{k}) =\bar{ x}_{n}\).
We have from Eq. (13) that \(\forall n \in \{\bar{ k},\ldots,l  1\}\), n is in freeflow and at maximum capacity under \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\) (i.e., \(\bar{x}_{n} = x_{n}^{\max }\) and \(\ell_{n}(\bar{x}_{n}) = a_{n}\)). Thus \(\forall n \in \{\bar{ k},\ldots,l  1\}\), \(\ell_{n}(x_{n},m_{n}) \geq a_{n} =\ell _{n}(\bar{x}_{n},\bar{m}_{n})\) and \(x_{n} \leq x_{n}^{\max } =\bar{ x}_{n}\). This completes the proof of the Lemma.
Therefore the NCF strategy is an optimal Stackelberg strategy, and it can be computed in polynomial time since it is generated in linear time after computing the best Nash equilibrium BNE(N, (1 −α)r), which can be computed in O(N^{2}).
The NCF strategy is, in general, not the unique optimal Stackelberg strategy. In the next section, we show that any optimal Stackelberg strategy can in fact be easily expressed in terms of the NCF strategy.
4.2 The Set of Optimal Stackelberg Strategies
In this section, we show that the set of optimal Stackelberg strategies S^{⋆}(N, r, α) can be generated from the NCF strategy. This shows in particular that the NCF strategy is robust, in a sense explained below.
Let \(\bar{\boldsymbol{s}} = \text{NCF}(N,r,\alpha )\) be the noncompliant first strategy, \(\{(\bar{\boldsymbol{t}},\bar{\boldsymbol{m}})\} = \text{BNE}(N,(1\alpha )r)\) be the Nash equilibrium induced by \(\bar{\boldsymbol{s}}\), and \(\bar{k} =\max \text{supp}\left (\bar{\boldsymbol{t}}\right )\) the last link in the support of the induced equilibrium, as defined above. By definition, the NCF strategy \(\bar{\boldsymbol{s}}\) assigns zero compliant flow to links \(\left \{1,\ldots,\bar{k}  1\right \}\) and saturates links one by one, starting from \(\bar{k}\) (see Eq. (7) and Fig. 4).
To give an example of an optimal Stackelberg strategy other than the NCF strategy, consider a strategy \(\boldsymbol{s}\) defined by \(\boldsymbol{s} =\bar{\boldsymbol{ s}}+\boldsymbol{\varepsilon }\), where
Lemma 5
Consider a Stackelberg strategy \(\boldsymbol{s}\) of the form \(\boldsymbol{s} =\bar{\boldsymbol{ s}}+\boldsymbol{\varepsilon }\) , where
Proof
We show that \(\boldsymbol{s} =\bar{\boldsymbol{ s}}+\boldsymbol{\varepsilon }\) is a feasible assignment of the compliant flow αr and that the induced equilibrium of the followers is \((\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s})) = (\bar{\boldsymbol{t}}\boldsymbol{\varepsilon },\bar{\boldsymbol{m}})\).

\(\forall n \in \{ 1,\ldots,\bar{k}  1\}\), \(s_{n} =\varepsilon _{n} \in [0,\hat{x}_{n}(\bar{k})]\) by Eq. (19). Thus s_{ n } ∈ [0, x_{ n }^{max}].

\(s_{\bar{k}} =\bar{ s}_{\bar{k}} +\varepsilon _{\bar{k}} \geq 0\) by Eq. (20), and \(s_{\bar{k}} \leq \bar{ s}_{\bar{k}} \leq x_{\bar{k}}^{\max }\).

\(\forall n \in \{\bar{ k} + 1,\ldots,N\}\), \(s_{n} =\bar{ s}_{n} \in [0,x_{n}^{\max }]\).
This shows that the NCF strategy is robust to perturbations: even if the strategy \(\bar{\boldsymbol{s}}\) is not realized exactly, it may still be optimal if the perturbation \(\boldsymbol{\varepsilon }\) satisfies the conditions given above.
The converse of the previous lemma is true. This gives a necessary and sufficient condition for optimal Stackelberg strategies, given in the following theorem.
Theorem 2 (Characterization of optimal Stackelberg strategies)
The set of optimal Stackelberg strategies S^{⋆}(N, r, α) is the set of strategies \(\boldsymbol{s}\) of the form \(\boldsymbol{s} =\bar{\boldsymbol{ s}}+\boldsymbol{\varepsilon }\) where \(\bar{\boldsymbol{s}} = {\text{NCF}}(N,r,\alpha )\) is the noncompliant first strategy, and \(\boldsymbol{\varepsilon }\) satisfies Eqs. (18), (19), and (20).
Proof
We prove the converse of Lemma 5. Let \(\boldsymbol{s} \in \text{S}^{\star }(N,r,\alpha )\) be an optimal Stackelberg strategy, \((\boldsymbol{t},\boldsymbol{m}) = (\boldsymbol{t}(\boldsymbol{s}),\boldsymbol{m}(\boldsymbol{s}))\) the equilibrium of noncompliant flow induced by \(\boldsymbol{s}\), \(k =\max \text{supp}\left (\boldsymbol{t}\right )\) the last link in the support of \(\boldsymbol{t}\), and \(\boldsymbol{x} =\boldsymbol{ s} +\boldsymbol{ t}\) the total flow assignment.
Let \(n \in \{ 1,\ldots,l  1\}\). From the expression (14) of the latencies under \(\bar{\boldsymbol{x}}\), we have \(\ell_{n}(\bar{x}_{n},\bar{m}_{n}) <a_{l}\); thus from equality (24), we have \(x_{n} \bar{ x}_{n} = 0\). Now let \(n \in \left \{l + 1,\ldots N\right \}\). We have by definition of the latency functions, ℓ_{ n }(x_{ n }, m_{ n }) ≥ a_{ n } > a_{ l }, thus from equality (23), x_{ n } = 0. We also have from the expression (13), \(\bar{x}_{n} = 0\). Therefore \(x_{n} =\bar{ x}_{n}\) \(\forall n\neq l\), but since \(\boldsymbol{x}\) and \(\bar{\boldsymbol{x}}\) are both assignments of the same total flow r, we also have \(x_{l} =\bar{ x}_{l}\), which proves \(\boldsymbol{x} =\bar{\boldsymbol{ x}}\).
Next we show that \(k =\bar{ k}\). We have from the proof of Theorem 1 that \(k \geq \bar{ k}\). Assume by contradiction that \(k>\bar{ k}\). Then since \(k \in \text{supp}\left (\boldsymbol{t}\right )\), we have by definition of the induced followers’ assignment in Eq. (5), \(\forall n \in \{ 1,\ldots,N\}\), ℓ_{ n }(x_{ n }, m_{ n }) ≥ ℓ_{ k }(x_{ k }, m_{ k }). And since \(\ell_{k}(x_{k},m_{k}) \geq a_{k}> a_{\bar{k}}\), we have (in particular for \(n =\bar{ k}\)) \(\ell_{\bar{k}}(x_{\bar{k}},m_{\bar{k}})> a_{\bar{k}}\), i.e., link \(\bar{k}\) is congested under \((\bar{\boldsymbol{x}},\bar{\boldsymbol{m}})\), thus \(x_{\bar{k}}> 0\). Finally, since \(\ell_{\bar{k}}(\bar{x}_{\bar{k}},\bar{m}_{\bar{k}}) = a_{\bar{k}}\), we have \(\ell_{\bar{k}}(\bar{x}_{\bar{k}},\bar{m}_{\bar{k}})>\ell _{\bar{k}}(\bar{x}_{\bar{k}},\bar{m}_{\bar{k}})\). Therefore \(x_{\bar{k}}(\ell_{\bar{k}}(x_{\bar{k}},m_{\bar{k}}) \ell_{\bar{k}}(\bar{x}_{\bar{k}},\bar{m}_{\bar{k}}))> 0\), since \(\bar{k} <k \leq l\); this contradicts (22).
Now let \(\boldsymbol{\varepsilon }=\boldsymbol{ s} \bar{\boldsymbol{ s}}\). We want to show that \(\boldsymbol{\varepsilon }\) satisfies Eq. (18), (19), and (20).
First, we have \(\forall n \in \left \{1,\ldots,\bar{k}  1\right \}\), \(\bar{s}_{n} = 0\), thus \(\varepsilon _{n} = s_{n} \bar{ s}_{n} = s_{n}\). We also have \(\forall n \in \left \{1,\ldots,\bar{k}  1\right \}\), 0 ≤ s_{ n } ≤ x_{ n }, \(x_{n} =\bar{ x}_{n}\) (since \(\boldsymbol{x} =\bar{\boldsymbol{ x}}\)), and \(\bar{x}_{n} =\hat{ x}_{n}(\bar{k})\) (by Eq. (13)); therefore \(0 \leq s_{n} \leq \hat{ x}_{n}(\bar{k})\). This proves (19).
Second, we have \(\forall n \in \left \{\bar{k} + 1,\ldots,N\right \}\), \(t_{n} =\bar{ t}_{n} = 0\) (since \(k =\bar{ k}\)), and \(x_{n} =\bar{ x}_{n}\) (since \(\boldsymbol{x} =\bar{\boldsymbol{ x}}\)); thus \(\varepsilon _{n} = s_{n} \bar{ s}_{n} = x_{n}  t_{n} \bar{ x}_{n} +\bar{ t}_{n} = 0\). We also have ∑_{n = 1}^{ N }ɛ_{ n } = 0 since \(\boldsymbol{s}\) and \(\bar{\boldsymbol{s}}\) are assignments of the same compliant flow αr; thus \(\varepsilon _{\bar{k}} = \sum _{n\neq \bar{k}}\varepsilon _{n} = \sum _{n=1}^{\bar{k}1}\varepsilon _{n}\). This proves (18).
Finally, we readily have (20) since \(s_{\bar{k}} \geq 0\) by definition of \(\boldsymbol{s}\).
5 Price of Stability Under Optimal Stackelberg Routing
To quantify the inefficiency of Nash equilibria, and the improvement that can be achieved using Stackelberg routing, several metrics have been used including price of anarchy (Roughgarden and Tardos 2002, 2004) and price of stability (Anshelevich et al. 2004). We use price of stability as a metric, which is defined as the ratio between the cost of the best Nash equilibrium and the cost of the social optimum.^{4} We start by characterizing the social optimum.
5.1 Characterization of Social Optima
Proposition 5
\((\boldsymbol{x}^{\star },\boldsymbol{m}^{\star })\) is optimal for (SO) only if \(\forall n \in \{ 1,\ldots,N\}\), m_{ n }^{⋆} = 0.
Proof
This follows immediately from the fact the latency on a link in congestion is always greater than the latency of the link in freeflow ℓ_{ n }(x_{ n }, 1) > ℓ_{ n }(x_{ n }, 0) \(\forall x_{n} \in (0,x_{n}^{\max })\).
5.2 Price of Stability and Value of Altruism
This terminology refers to the improvement achieved by having a fraction α of altruistic (or compliant) players, compared to a situation where everyone is selfish. We give the expressions of price of stability and value of altruism in the case of a twolink network, as a function of the compliance rate α ∈ [0, 1] and demand r.
5.2.1 Case 1: 0 ≤ (1 −α)r ≤ x_{1}^{max}
5.2.2 Case 2: \(x_{1}^{\max } <(1\alpha )r \leq x_{2}^{\max } +\hat{ x}_{1}(2)\)
We observe that for a fixed flow demand r > x_{1}^{max}, the price of stability is an increasing function of a_{2}∕a_{1}. Intuitively, the inefficiency of Nash equilibria increases when the difference in freeflow latency between the links increases. And as a_{2} → a_{1}, the price of stability goes to 1.
Therefore optimal Stackelberg routing can significantly decrease price of stability when r ∈ (x_{1}^{max}, x_{1}^{max}∕(1 −α)). This can occur for small values of the compliance rate in situations where the demand slightly exceeds the capacity of the first link (Fig. 8c).
The same analysis can be done for a general network: given the latency functions on the links, one can compute the price of stability as a function of the flow demand r and the compliance rate α, using the form of the NCF strategy together with Algorithm 1 to compute the BNE. Computing the price of stability function reveals critical values of demand, for which optimal Stackelberg routing can lead to a significant improvement. This is discussed in further detail in the next section, using an example network with four links.
6 Numerical Results
It can be shown that r^{(α)}(n) = r^{NE}(n)∕(1 −α), and we have in particular r^{NE}(n) = r^{(0)}(n). Therefore if a link n is congested under best Nash equilibrium (r > r^{NE}(n)), optimal Stackelberg routing can decongest n if r^{(α)}(n) ≥ r. In particular, when the demand is slightly above critical demand r^{(0)}(n), link n can be decongested with a small compliance rate. This is illustrated by the numerical values of price of stability on Fig. 11a, where a small compliance rate (α = 0. 05) achieves high value of altruism when the demand is slightly above the critical values. This shows that optimal Stackelberg routing can achieve a significant improvement in efficiency, especially when the demand is near one of the critical values r^{(α)}(n).
7 Summary and Concluding Remarks
Motivated by the fundamental diagram of traffic for transportation networks, this chapter has introduced a new class of latency functions (HQSF) to model congestion with horizontal queues and studied the resulting Nash equilibria for nonatomic routing games on parallel networks. We showed that the essential uniqueness property does not hold for HQSF latencies and that the number of equilibria is at most 2N. We also characterized the best Nash equilibrium. In the Stackelberg routing game, we proved that the noncompliant first (NCF) strategy is optimal and that it can be computed in polynomial time. Table 1 summarizes the main differences between the classical setting (vertical queues) and the HQSF setting.
Main assumptions and results for the Stackelberg routing game on a parallel network
Setting  Vertical queues  Horizontal queues, single valued in freeflow (HQSF)  

Model  x ↦ ℓ(x)  (x, m) ↦ ℓ(x, m)  
latency is a function of the flow x ∈ [0, x^{max}]  latency is a function of the flow x ∈ [0, x^{max}] and the congestion state m ∈ {0, 1}  
Assumptions  x ↦ ℓ(x) is continuously nondecreasing  x ↦ ℓ(x, 0) is single valued. x ↦ ℓ(x, 1) is continuously decreasing  
x ↦ xℓ(x) is convex  \(\lim _{x\rightarrow x^{\max }}\ell(x,1) =\ell (x^{\max },0)\)  
Set of Nash equilibria  Essential uniqueness: if x, x′ are Nash equilibria, then C(x) = C(x′) (Beckmann et al. 1956)  No essential uniqueness in general  
The number of Nash equilibria is at most 2N (Proposition 4)  
The best Nash equilibrium is a singlelinkfreeflow equilibrium (Lemma 2)  
Optimal Stackelberg strategy  NP hard (Roughgarden 2001)  The NCF strategy is optimal and can be computed in polynomial time (Theorem 1)  
The set of all optimal Stackelberg strategies can be computed in polynomial time (Theorem 2) 
We illustrated these results using an example network for which we computed the decrease in inefficiency that can be achieved using optimal Stackelberg routing. This example showed that when the demand is near critical values r^{NE}(n), optimal Stackelberg routing can achieve a significant improvement in efficiency, even for small values of compliance rate.
On the one hand, these results show that careful routing of a small compliant population can dramatically improve the efficiency of the network. On the other hand, they also indicate that for certain demand and compliance values, Stackelberg routing can be completely ineffective. Therefore identifying the ranges where optimal Stackelberg routing does improve the efficiency of the network is crucial for effective planning and control.
This framework offers several directions for future research: the work presented here only considers parallel networks under static assumptions (constant flow demand r and static equilibria), and one question is whether these equilibria are stable in the dynamic sense and how one may steer the system from one equilibrium to a better one – consider, for example, the case where the players are in a congested equilibrium and assume a coordinator has control over a fraction of the flow. Can the coordinator steer the system to a singlelinkfreeflow equilibrium by decongesting a link? And what is the minimal compliance rate needed to achieve this?
Footnotes
 1.
The latency in congestion ℓ_{ n }(⋅ , 1) is defined on the open interval (0, x_{ n }^{max}). In particular, if x_{ n } = 0 or x_{ n } = x_{ n }^{max} then the link is always considered to be in freeflow. When the link is empty (x_{ n } = 0), it is naturally in freeflow. When it is at maximum capacity (x_{ n } = x_{ n }^{max}) it is in fact on the boundary of the freeflow and congestion regions, and we say by convention that the link is in freeflow.
 2.
We note that a feasible flow assignment \(\boldsymbol{s}\) of compliant flow may fail to induce a Nash equilibrium \((\boldsymbol{t},\boldsymbol{m})\) and therefore is not considered to be a valid Stackelberg strategy.
 3.
 4.
Price of anarchy is defined as the ratio between the costs of the worst Nash equilibrium and the social optimum. For the case of nondecreasing latency functions, the price of anarchy and the price of stability coincide since all Nash equilibria have the same cost by the essential uniqueness property.
References
 Anshelevich E, Dasgupta A, Kleinberg J, Tardos E, Wexler T, Roughgarden T (2004) The price of stability for network design with fair cost allocation. In: 45th annual IEEE symposium on foundations of computer science, Berkeley, pp 295–304Google Scholar
 Aswani A, Tomlin C (2011) Gametheoretic routing of GPSassisted vehicles for energy efficiency. In: American control conference (ACC). IEEE, San Francisco, pp 3375–3380Google Scholar
 Babaioff M, Kleinberg R, Papadimitriou CH (2009) Congestion games with malicious players. Games Econ Behav 67(1):22–35MathSciNetCrossRefzbMATHGoogle Scholar
 Beckmann M, McGuire CB, Winsten CB (1956) Studies in the economics of transportation. Yale University Press, New HavenGoogle Scholar
 Benaïm M (2015) On gradient like properties of population games, learning models and self reinforced processes. Springer, Cham, pp 117–152Google Scholar
 Blackwell D (1956) An analog of the minimax theorem for vector payoffs. Pac J Math 6(1):1–8MathSciNetCrossRefzbMATHGoogle Scholar
 Blum A, EvenDar E, Ligett K (2006) Routing without regret: on convergence to Nash equilibria of regretminimizing algorithms in routing games. In: Proceedings of the twentyfifth annual ACM symposium on principles of distributed computing (PODC’06). ACM, New York, pp 45–52CrossRefGoogle Scholar
 Blume LE (1993) The statistical mechanics of strategic interaction. Games Econ Behav 5(3):387–424MathSciNetCrossRefzbMATHGoogle Scholar
 Boulogne T, Altman E, Kameda H, Pourtallier O (2001) Mixed equilibrium for multiclass routing games. IEEE Trans Autom Control 47:58–74MathSciNetGoogle Scholar
 Caltrans (2010) US 101 South, corridor system management plan. http://www.dot.ca.gov/hq/tpp/offices/ocp/pp_files/new_ppe/corridor_planning_csmp/CSMP_outreach/US101south/2_US101_south_CSMP_Exec_Summ_final_22511.pdf
 CesaBianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
 Dafermos S (1980) Traffic equilibrium and variational inequalities. Transp Sci 14(1):42–54MathSciNetCrossRefGoogle Scholar
 Dafermos SC, Sparrow FT (1969) The traffic assignment problem for a general network. J Res Natl Bur Stand 73B(2):91–118MathSciNetCrossRefzbMATHGoogle Scholar
 Daganzo CF (1994) The cell transmission model: a dynamic representation of highway traffic consistent with the hydrodynamic theory. Transp Res B Methodol 28(4):269–287CrossRefGoogle Scholar
 Daganzo CF (1995) The cell transmission model, part II: network traffic. Transp Res B Methodol 29(2):79–93CrossRefGoogle Scholar
 Drighès B, Krichene W, Bayen A (2014) Stability of Nash equilibria in the congestion game under replicator dynamics. In: 53rd IEEE conference on decision and control (CDC), Los Angeles, pp 1923–1929Google Scholar
 Evans CL (1998) Partial differential equations. Graduate studies in mathematics. American Mathematical Society, ProvidencezbMATHGoogle Scholar
 Farokhi F, Johansson HK (2015) A piecewiseconstant congestion taxing policy for repeated routing games. Transp Res B Methodol 78:123–143CrossRefGoogle Scholar
 Fischer S, Vöcking B (2004) On the evolution of selfish routing. In: Algorithms–ESA 2004. Springer, Berlin, pp 323–334CrossRefGoogle Scholar
 Fischer S, Räcke H, Vöcking B (2010) Fast convergence to Wardrop equilibria by adaptive sampling methods. SIAM J Comput 39(8):3700–3735MathSciNetCrossRefzbMATHGoogle Scholar
 Fox MJ, Shamma JS (2013) Population games, stable games, and passivity. Games 4(4):561–583MathSciNetCrossRefzbMATHGoogle Scholar
 Friesz TL, Mookherjee R (2006) Solving the dynamic network user equilibrium problem with statedependent time shifts. Transp Res B Methodol 40(3):207–229CrossRefGoogle Scholar
 Greenshields BD (1935) A study of traffic capacity. Highw Res Board Proc 14:448–477Google Scholar
 Hannan J (1957) Approximation to Bayes risk in repeated plays. Contrib Theory Games 3:97–139MathSciNetzbMATHGoogle Scholar
 Hofbauer J, Sandholm WH (2009) Stable games and their dynamics. J Econ Theory 144(4):1665–1693.e4Google Scholar
 Kleinberg R, Piliouras G, Tardos E (2009) Multiplicative updates outperform generic noregret learning in congestion games. In: Proceedings of the 41st annual ACM symposium on theory of computing, Bethesda, pp 533–542. ACMGoogle Scholar
 Korilis YA, Lazar AA, Orda A (1997a) Achieving network optima using Stackelberg routing strategies. IEEE/ACM Trans Netw 5:161–173CrossRefGoogle Scholar
 Korilis YA, Lazar AA, Orda A (1997b) Capacity allocation under noncooperative routing. IEEE Trans Autom Control 42:309–325MathSciNetCrossRefzbMATHGoogle Scholar
 Koutsoupias E, Papadimitriou C (1999) Worstcase equilibria. In: Proceedings of the 16th annual conference on theoretical aspects of computer science. Springer, Heidelberg, pp 404–413Google Scholar
 Krichene S, Krichene W, Dong R, Bayen A (2015a) Convergence of heterogeneous distributed learning in stochastic routing games. In: 53rd annual allerton conference on communication, control and computing, MonticelloGoogle Scholar
 Krichene W, Drighès B, Bayen A (2015b) Online learning of Nash equilibria in congestion games. SIAM J Control Optim (SICON) 53(2):1056–1081MathSciNetCrossRefzbMATHGoogle Scholar
 Krichene W, Castillo MS, Bayen A (2016) On social optimal routing under selfish learning. IEEE Trans Control Netw Syst PP(99):1–1Google Scholar
 Lam K, Krichene W, Bayen A (2016) On learning how players learn: estimation of learning dynamics in the routing game. In: 7th international conference on cyberphysical systems (ICCPS), ViennaGoogle Scholar
 Lebacque JP (1996) The Godunov scheme and what it means for first order traffic flow models. In: International symposium on transportation and traffic theory, Lyon, pp 647–677Google Scholar
 LeVeque RJ (2007) Finite difference methods for ordinary and partial differential equations: steadystate and timedependent problems. Classics in applied mathematics. Society for Industrial and Applied Mathematics, PhiladelphiaCrossRefzbMATHGoogle Scholar
 Lighthill MJ, Whitham GB (1955) On kinematic waves. II. A theory of traffic flow on long crowded roads. Proc R Soc Lond A Math Phys Sci 229(1178):317Google Scholar
 Lo HK, Szeto WY (2002) A cellbased variational inequality formulation of the dynamic user optimal assignment problem. Transp Res B Methodol 36(5):421–443CrossRefGoogle Scholar
 Monderer D, Shapley LS (1996) Potential games. Games Econ Behav 14(1):124–143MathSciNetCrossRefzbMATHGoogle Scholar
 Ozdaglar A, Srikant R (2007) Incentives and pricing in communication networks. In: Nisan N, Roughgarden T, Tardos E, Vazirani V (eds) Algorithmic game theory. Cambridge University Press, CambridgeGoogle Scholar
 Papageorgiou M, Blosseville JM, HadjSalem H (1989) Macroscopic modelling of traffic flow on the Boulevard Périphérique in Paris. Transp Res B Methodol 23(1):29–47CrossRefGoogle Scholar
 Papageorgiou M, Blosseville J, HadjSalem H (1990) Modelling and realtime control of traffic flow on the southern part of boulevard peripherique in paris: part I: modelling. Transp Res A Gen 24(5):345–359CrossRefGoogle Scholar
 Richards PI (1956) Shock waves on the highway. Oper Res 4(1):42–51MathSciNetCrossRefGoogle Scholar
 Roughgarden T (2001) Stackelberg scheduling strategies. In: Proceedings of the thirtythird annual ACM symposium on theory of computing, Heraklion, pp 104–113. ACMGoogle Scholar
 Roughgarden T, Tardos E (2002) How bad is selfish routing? J ACM (JACM) 49(2):236–259MathSciNetCrossRefzbMATHGoogle Scholar
 Roughgarden T, Tardos E (2004) Bounding the inefficiency of equilibria in nonatomic congestion games. Games Econ Behav 47(2):389–403MathSciNetCrossRefzbMATHGoogle Scholar
 Sandholm WH (2001) Potential games with continuous player sets. J Econ Theory 97(1):81–108MathSciNetCrossRefzbMATHGoogle Scholar
 Sandholm WH (2010) Population games and evolutionary dynamics. MIT Press, CambridgezbMATHGoogle Scholar
 Schmeidler D (1973) Equilibrium points of nonatomic games. J Stat Phys 7(4):295–300MathSciNetCrossRefzbMATHGoogle Scholar
 Shamma JS (2015) Learning in games. Springer, London, pp 620–628Google Scholar
 Swamy C (2007) The effectiveness of Stackelberg strategies and tolls for network congestion games. In: Proceedings of the eighteenth annual ACMSIAM symposium on discrete algorithms (SODA’01), New Orleans. Society for Industrial and Applied Mathematics, Philadelphia, pp 1133–1142Google Scholar
 Thai J, Hariss R, Bayen A (2015) A multiconvex approach to latency inference and control in traffic equilibria from sparse data. In: American control conference (ACC), Chicago, pp 689–695Google Scholar
 Wang Y, Messmer A, Papageorgiou M (2001) Freeway network simulation and dynamic traffic assignment with METANET tools. Transp Res Rec J Transp Res Board 1776(1):178–188Google Scholar
 Wardrop JG (1952) Some theoretical aspects of road traffic research. In: ICE proceedings: engineering divisions, vol 1, pp 325–362Google Scholar
 Weibull JW (1997) Evolutionary game theory. MIT Press, CambridgezbMATHGoogle Scholar
 Work DB, Blandin S, Tossavainen OP, Piccoli B, Bayen AM (2010) A traffic model for velocity data assimilation. Appl Math Res eXpress 2010(1):1MathSciNetzbMATHGoogle Scholar