A Numerical Algorithm to Calculate the Unique Feedback Nash Equilibrium in a Large Scalar LQ Differential Game

In this paper, we study scalar linear quadratic differential games with state feedback information structure. We present a numerical algorithm which determines whether this game will have no, one, or multiple equilibria. Furthermore, in case there is a unique equilibrium, the algorithm provides this equilibrium. The algorithm is efficient in the sense that it is capable of handling a large number of players. The analysis is restricted to the case the involved cost depend only on the state and control variables.


Introduction
A wide range of settings like communication, environmental, macroeconomic, epidemiological, transportation, and energy systems are characterized by dynamical systems in which large populations interact either in a competitive or cooperative way.In the last decades, there is an increased interest to study the interaction within these systems using the framework of dynamic games, that is, by an appropriate modeling of the underlying dynamics of the system, the objective function of the involved agents, and specification of the information structure (including often the specification of some assumed equilibrium concept).In environmental economics and macroeconomic policy coordination, references and examples of using dynamic games in modeling policy coordination problems can be found, e.g., in the books of [6,15,18,30]).In engineering, applications of this theory are reported, e.g., in areas like finance, robust optimal control, and pursuit-evasion problems.Particularly in the area of robust optimal control, which can be modeled as a problem where the controller is fighting "nature" producing worst-case disturbances, the theory of linear quadratic differential games B Jacob Engwerda engwerda@uvt.nl 1 Department of Econometrics and Operations Research, Tilburg University, P.O.Box: 90153, 5000 LE Tilburg, The Netherlands has been extensively developed, see, e.g., [3,8,20,26].In engineering, using this framework, applications are reported from diverse areas: robot control formation [16]; interconnection of electric power systems [26]; multipath routing in communication networks [2,23]; solving mixed H 2 /H ∞ control problems [22]; and military operations of autonomous vehicles [21].Furthermore, in games involving a large number of players mean field approximation techniques are reported to arrive at decentralized strategies for the original game, which is then an -Nash equilibrium for the original game [17].
In linear quadratic differential games, "real world" is modeled/approximated by a set of linear differential equations and the objectives are modeled using quadratic functions.Assuming that players don't cooperate and look for linear feedback strategies which lead to a worse performance if they unilaterally deviate from it, leads to the study of so-called linear feedback Nash equilibria (FNE).The resulting equilibrium strategies have the important property that they are strong time consistent, a property which, e.g., does not hold under an open-loop information structure (see, e.g., [4,Chapter 6.5]).
Linear quadratic feedback Nash differential games have been considered by many authors and dates back to the seminal work of Starr and Ho [32].For the fixed finite planning horizon, there exists at most one FNE (see, e.g., [24]).For an infinite planning horizon, the affinequadratic differential game is solved in [13].To find the FNE in this game involves solving a set of coupled algebraic Riccati-type equations (ARE).Only for some special cases of these equations conditions are reported under which FNE exist (see, e.g., [1,8,29]).It has been shown (see, e.g., [8,28]) that the number of equilibria can vary between zero and infinity.Many numerical approaches are reported in the literature to find a solution of the ARE (see for an overview, e.g., [10]).Usually, these approaches only find one solution (if convergence occurs) and it remains unclear whether more solutions exist.More recently in [11] and [31], also algorithms are reported which, in principle, are capable of finding all solutions.These algorithms seem to work if the number of players is not too large.Engwerda [11] uses an eigenvalue-based approach to find all solutions for the general scalar game, whereas [31] uses techniques from algebraic geometry to recast the problem of computing all FNE into that of finding the zeros of a single polynomial function in a scalar variable.
Particularly in the context of large-scale systems, it seems interesting to have a decisive algorithm to conclude whether no, or, a unique equilibrium exists and to calculate such a solution if it exists within a reasonable computation time.Separate from direct applications, this may be helpful, e.g., in the assessment of calculating the cost/gains of different information/cooperation structures (see, e.g., [5]), or, in finding areas where approximate solutions of certain nonlinear differential games exist (see, e.g., [27]).Following the analysis performed in [7] (see also [8,Chapter 8.4]) in [12], an exhaustive description is given of conditions under which the most simple linear scalar game, where the performance criterion is a strict positive quadratic function of both states and controls, has either no, one, or multiple FNE.In this paper, we use this approach to present a numerical algorithm which is capable of answering for these games the question whether there will exist a unique FNE.And, in case a unique FNE exists, to calculate it.Using standard MATLAB code, we show in a simulation study that for systems having up to 100.000 players the FNE can be calculated within 1 s.This, since the main body of the algorithm just requires the calculation of the zero of a scalar function, where the bounds of the search interval are known.
So, whereas, e.g., [9] and [11] concentrate on finding all FNE equilibria in general scalar (disturbed) games with a small number of players (due to computation time constraints), this paper at first instance concentrates on the question whether a special subclass of these games with a large number of players has a unique equilibrium.Furthermore, in case this question is answered affirmative, this equilibrium can be calculated using, e.g., a brute-force halving technique which makes that computation time is small.For instance, in the example dealt with in Sect.5, calculation time of the equilibrium (for a reasonable accuracy) remains below 1 s for a game that has 100.000players.The numerical algorithms that are established use the conditions derived in [12].They will be summarized in Sect.3.
The outline of the paper is as follows.Section 2 recalls from [8] the basic model and approach.In Sect.3, we recall from [12] the conditions under which the game will have either no, one, or multiple equilibria.Based on the results presented in Sects. 2 and 3, we develop in Sect. 4 numerical algorithms to calculate the unique FNE.Details on proofs of Sect. 4 are presented in the separate "Appendix 1".Section 5 illustrates the algorithm for a simple oligopoly game.Finally, Sect.6 concludes.

Preliminaries
In this paper, we consider the problem where N players try to minimize their performance criterion in a noncooperative setting.Each player controls a different set of inputs to a single system.The system is described by the following scalar differential equation Here x is the state of the system, u i is a (control) variable player i can manipulate, x 0 is the arbitrarily chosen initial state of the system, a (the state feedback parameter), b i , i ∈ N: = {1, . . ., N }, are constant system parameters, and ẋ denotes the time derivative of x.All variables are scalar.The aim of player i ∈ N is to minimize: where r i is positive and both b i and q i differ from zero.So, player i is not directly concerned about the control efforts player j uses to manipulate the system.This assumption is crucial for the analysis below.We assume that players act noncooperatively and use time invariant feedback strategies, u i (t) = f i x(t), to control the system.This, on the supposition that they do not want to destabilize the system.So, the set of strategies is restricted to This restriction is essential.Indeed, there exist feedback Nash equilibria in which a player can improve unilaterally by choosing a feedback for which the closed-loop system is unstable (see [25]).Any f ∈ F is called a stabilizing solution.A set of feedback strategies is called a Nash equilibrium if none of the players can improve his performance by unilaterally deviating from his strategy within this class of stationary stabilizing feedback controls.More formally, using the notation f−i ( f i ): = ( f1 , . . ., fi−1 , f i , fi+1 , . . ., fN ): The N -tuple f : = ( f1 , . . ., fN ) is called a set of (linear stabilizing stationary) feedback Nash equilibrium strategies if for all i ∈ N the following inequalities hold: for all initial states x 0 and for all In the sequel, we will drop the adjectives linear, stabilizing and stationary in the above definition and use the shorthand notation FNE to denote the by these actions implied equilibrium cost, and the actions themselves as FNE actions or strategies.
In this problem setting, we may assume, without loss of generality, that r i are positive and both b i and q i differ from zero.This, since in case r i ≤ 0 the problem has no solution; and in case either b i = 0 or q i = 0 the optimal control for player i is to use no control, i.e., u i (•) = 0, at any point in time.So, in the last-mentioned case, the player could be discarded from the game.For this game, we distinguish three cases.
Definition 2.2 Consider the cost function (2).The game is called an economic game if q i < 0, i ∈ N; a regulator game if q i > 0, i ∈ N; a mixed game if for some indices q i is negative, and for other indices this parameter is positive.
The attached names are inspired by the fact that, in case q i < 0, i ∈ N, the game can be interpreted as a game between players who all like to maximize their profits (measured by the state variable x) using their input (measured by u i ) as efficient as possible.Whereas in case q i > 0, i ∈ N, the game can be interpreted as a problem where all players like to track the system's state, x, as fast as possible to zero using as less as possible control efforts, u i .
The FNE for the game (1, 2) are completely characterized by the solutions of a set of coupled algebraic Riccati equations (ARE).With s i : = b 2 i r i , these equations in the variables k i reduce to (see, e.g., [8]): The precise statement is as follows: Theorem 2. 3 The game (1,2) has a FNE if and only if (iff.) there exist N scalars k i such that (3) holds and a − N j=1 s j k j < 0. If this condition holds, the N -tuple ( f1 , . . ., fN ) with fi : = −r −1 i b i k i is a FNE and J i ( f1 , . . ., fN , x 0 ) = k i x 2 0 .So, to determine the set of FNE we have to find all stabilizing solutions of (3).To determine these solutions, following [8, Section 8.5.1], we introduce (for notational convenience) the variables: Note that, by relabeling the player indices, we can enforce that σ 1 ≥ • • • ≥ σ N .This ordering is assumed to hold throughout.Furthermore, since f i = −1 b i y i , there is a bijection between ( f 1 , . . ., f N ) and (y 1 , . . ., y N ).Using this notation, (3) can be rewritten as The above problem can then be reformulated as: under which conditions have the above N quadratic equations, and the equation a real solution y i , i ∈ N, with y N +1 > 0.

Remark 2.4
In the rest of this paper, we concentrate on providing numerical algorithms which provide a conclusive answer whether (3,5) have either no, one, or more than one real solution y i , i ∈ N, with y N +1 > 0. We will distinguish three algorithms, based on the sign of the involved σ i parameters.Since there is a one-to-one correspondence between the signs of parameter q i in performance criterion (2) and the sign of parameter σ i , we will use the notation introduced in Definition 2.2 for these equations.
As we will see later on in Sect. 5 on the oligopolistic competition example, the determination of the unique solutions to these Eqs.(3,5) enables to determine, e.g., FNE for different/higher-dimensional systems too.In those cases, the interpretation in terms of the original scalar game is, of course, only indirect.

Solvability Conditions
Theorem 3.3 below presents conditions under which the game will have either none, precisely one, or, multiple equilibria.The results are obtained by a detailed study of the set of functions where In particular, next three functions from this set F play a crucial role.
Geometrically, the number of FNE is obtained by counting, for a fixed level a, the number of intersection points with all functions from F .From [12, Lemma A.1], we recall next properties of above-mentioned functions.
Above Lemma 3.1 shows that for all a ≥ f 3 (0) the game will have more than one equilibrium.Consequently, a detailed study of the functions f i (x), i = 1, 2, 3 provides complete information on conditions under which the game will have either no, one, or more than one equilibrium.This study is performed in [12].To create some intuition for the behavior of these functions, Lemma 3.2 recalls some general properties concerning the functions f i (x), i = 1, 2, 3.The next subsections report existence conditions from that study.
Lemma 3.2 1.In the regulator game: has at most two stationary points x 1 and x 2 ≥ x 1 , yielding a local maximum at x 1 and a minimum (which might be global) at has at most one stationary point, yielding a global minimum.

In the economic game:
(i) f 1 (x) has exactly one stationary point x * > 0, where it attains a (global) maximum.

In the mixed game:
(i) f 1 (x) has at most two stationary points, yielding a local minimum and a maximum (which might be global), respectively (and an inflection point if there is just one stationary point).(ii) f 2 (x) and f 3 (x) have the same properties as in item 1.

The 2-Player Case
For the 2-player case the question, for which values of the state feedback parameter a the game will have either no, one, or more than one FNE, is solved by calculating, for fixed a, the total number of solutions for the equations This, under the understanding that in the regulator and mixed game, only solutions x ≥ √ σ 1 are relevant, whereas for the economic game only solutions x ≥ 0 apply.By determining the various extremal points (including the boundary points) of the functions f i (x), i = 1, 2, 3, and using the monotonicity properties of these functions, one can characterize then the areas where either no, one, or more than one FNE will occur in terms of these extremal points.
Introducing the notation f i * , f * i for the minimum and maximum of function f i (x), respectively, Tables 1 and 2, below, present the results for the 2-player case.In this case, an analytic solution of the problem is possible.Table 1 reports how many equilibria exist for every choice of the state feedback parameter a for the generic case that σ 1 > σ 2 .For the case σ 1 = σ 2 , a separate analysis is required, as in that case the functions f 2 and f 3 coincide.Corresponding results for that case are displayed in Table 2 where With y as defined above, and, for the economic game (where In case the game is symmetric, i.e., σ 1 = σ 2 = σ , f 2 (x) = f 3 (x) = x.So, these functions are monotonically increasing.Furthermore, for the regulator game Due to this, it can be shown that at a = √ σ a unique equilibrium occurs.Consequently, In particular, this last observation implies that in the symmetric case in the economic game the interval ( f 2 (0), f 3 (0)) is empty.These observations give rise to Table 2.

The Symmetric Case
Next we consider the symmetric case if the number of players exceeds two.That is, the case that From this definition, it follows that in the symmetric case F reduces to the set of functions In particular, f 1 coincides with g 1 and both f 2 and f 3 coincide with g 2 .That is, It can be easily shown in this case that optima of f i (x), i = 1, 2, 3, occur at stationary points.Some elementary calculations show that Consequently, again, analytic conditions can be derived under which there exists either no, precisely one, or more than one equilibrium.Table 3 presents the results for the two games.

The General Case
In the previous two subsections, results were recalled where analytic solutions could be obtained.Unfortunately, this is not possible for the general case.The results advertised below show that the question whether the game will have no, one, or more than one equilibrium, basically, requires the calculation of the stationary points of the functions f i (x) i = 1, 2, 3.
The numerical algorithms developed in the next section use these results, together with estimates of these stationary points, to determine in a large number of cases efficiently whether the game will have no, one, or more than one equilibrium.And, in case there is a unique equilibrium, to calculate it.

Theorem 3.3 Consider the scalar game (1, 2).
1.In the regulator and mixed game, there always exists a FNE.Moreover, for every FNE strategy the closed-loop state feedback parameter a cl satisfies: a cl ≤ − √ σ 1 .

The economic game has no FNE
3. For very stable systems (i.e., a 0) all three games have a unique FNE.For very unstable systems (i.e., a 0), all three games have 2 N − 1 equilibria.
Theorem 3.4 Let f i (x), i = 1, 2, 3, be as defined in (7)(8)(9).Then, the items below present both necessary and sufficient conditions for the existence of a unique feedback Nash equilibrium for the considered games.

Consider the regulator game. Let S f
. This game has a unique equilibrium if a < f 1 (0).Furthermore,

Consider the mixed game with the notation from item 1.
(a) If f 1 (x) is monotonically decreasing, the existence conditions for the regulator game in item 1 apply.
Table 4 summarizes the above results.
Remark 3.5 1.In [12], it is shown that the sets S f i have at most two elements.If in the mixed game, case b., f 2 (x) is monotonically increasing, the corresponding maximum and minimum values are not defined.It is easily verified that the corresponding statements continue to hold if we define f 2 * = f * 2 = −∞ in that case.2. Since f 2 (x) has a global minimum and f 2 (x) ≤ f 3 (x), the global minimum of f 2 (x) is smaller than f 3 * .So, by Theorem 3.4, item 1., the regulator game has a unique FNE for all a smaller than the global minimum of f 2 (x). 3. From Theorem 3.4 and its proof, we obtain the next specializations in case (i) The regulator game has a unique FNE if a < f 3 * .Additionally, only a unique equilibrium occurs at The mixed game has a unique FNE iff., in case f 1 (x) is monotonically decreasing the conditions mentioned under item i) apply; and, in case Proposition 3.6 reports some sufficient analytic conditions from [12] under which the game has a unique FNE.
Proposition 3. 6 The game (1, 2) has a unique FNE in the following situations.
a.In the regulator game, with z 1 := b.In the economic game: for all a < − N i=1 √ −σ i .
c.In the mixed game: let z 2 := √ σ 1 and a 4 := a 0 , whereas for N 1 ≥ 2: i=2 σ i +a 0 .Then, the mixed game has a unique equilibrium: In particular, this results in the next observation for the regulator game.

Corollary 3.7
The regulator game has a unique equilibrium if a < 0.

Numerical Algorithms
In this section, we provide computational schemes to verify in an efficient way whether the game will have no, one, or more than one equilibrium.And, in case there is a unique equilibrium, to calculate this equilibrium.Below we present for each game separately the computational scheme.Proofs can be found in "Appendix 1".The setup of the algorithms is the same.First, there is a general check on the number of players and symmetry of the game.If the game has two (nonsymmetric) players, we use results from Sect.3.1 to present the solution.In case the game is symmetric, results from Sect.3.2 are used to calculate the solution.Next, for every game, we calculate numbers a * and a * , respectively.These numbers have the property that they can be easily calculated and determine large regions where either a unique equilibrium exists (for all state feedback parameters a smaller than a * ) or multiple equilibria exist (for all a > a * ).The third step in the algorithms use more game specific information to explore efficiently what is going on if a * ≤ a ≤ a * .In case there exists a unique equilibrium, the corresponding solution, k i , for the coupled algebraic Riccati Eq. ( 3) can be easily obtained from the intersection point of a with either f 1 (x) or f 2 (x).This follows from Lemma 2.5, item 1.In case f 1 (x) = a has a solution y, y i := y − y 2 − σ i , i ∈ N, solves (4, 5).Or, using the definition of y i , solves the set of coupled algebraic Riccati Eq. (3).Similarly, the solutions of (3) are obtained in case f 2 (x) = a has a solution y.The only solution that is obtained differently in that case is of y 1 which is then obtained as y 1 := y + y 2 − σ 1 .So, in that case the solutions of ( 3) are The corresponding equilibrium actions, fi = − b i k i r i , follow then directly from this (see Theorem 2.3).

The Regulator Game
Algorithm 4.1 1. 2-player case a. Calculate S and y from (10).b.If a < f 3 * (see Table 1), proceed with item c.Otherwise, there is no unique equilibrium.c.If a < f 1 ( √ σ 1 ), k i is given by (13).Else k i is given by ( 14).

Symmetric case
, see Tables 2 and 3, proceed with item b.Otherwise, there is no unique equilibrium.b. k i is given by ( 13).

General case
I. If a ≥ f 3 * , there exists more than one equilibrium.II.If f 1 ( √ σ 1 ) ≤ a < f 3 * : Calculate solution(s) of f 2 (y) = a on I .

Case i:
There exists more than one solution.Multiple equilibria exist.
Case ii: There exists no solution.k i is given by ( 14) where y > x solves a = f 2 (y) (see Remark 4.2.
Case i: There exists at least one solution.Multiple equilibria exist.
Case ii: There exists no solution.Go to item c. c. k i is given by (13) where y solves a = f 1 (y) (see Remark 4.2.1).
In Remark 4.2, we discuss some computational issues occurring in the above Algorithm 4.1.

Remark 4.2 1. To determine the solution of f
√ σ 1 ) > 0, and with x r := N i=1 σ i + |a|, F 1 (x r ) < 0. Using this, e.g., a brute-force halving technique or Newton-Raphson can be used to calculate the solution of F 1 (y) = 0. Other estimates that may be appropriate to narrow the initial search interval are, e.g.,  1).If a = f * 1 , proceed with item c.Otherwise, there is no unique equilibrium.c.If either a < f 1 (0) or a = f * 1 holds, k i is given by (13).Else k i is given by ( 14). 2. Symmetric case a.If either a < f 1 (0) or, in case N = 2, 3, or 4, a = f * 1 (see Tables 2, 3), proceed with item b.Otherwise, there is no unique equilibrium.b.If a < f 1 (0), y := and k i is given by (13).If either N = 2, 3 or 4 and −σ and k i is obtained again as (13).

General case
there is more than one equilibrium.If (14) where y ∈ [0, x r ] solves a = f 2 (y).Here (14) where y ∈ [0, x r ] solves a = f 2 (y), where x r is as in Case ii.If a = f 1 ( x), go to item c, with y := x.If f 1 (0) < a < f 2 (0), there exists no equilibrium.If a ≥ f 3 (0), multiple equilibria exist.c. k i is given by (13) where y solves a = f 1 (y) (see Remark 4.4.1).
Similar remarks we made in Remark 4.2 apply for the above algorithm here.Remark 4.4 1.To determine the solution of f 1 (y) = a in item 3.c, notice that F 1 (x) := f 1 (x) − a has a unique zero; F 1 (0) > 0; and F 1 (|a|) < 0. So, the search can be restricted to the interval [0, |a|].Clearly, the bounds of this interval are not tight.So, to improve calculation speed in certain applications, it might be worthwhile to improve on them.2. A similar remark as in item 1 applies w.r.t.F 2 (x) = f 2 (x) − a.Also here, in case N becomes large, it might be worthwhile from a computational point of view to improve the bound x r in item 3.b.

The Mixed Game
Algorithm 4.5 Following the notation of Proposition 3.6, let σ i > 0, i ∈ N 1 (N 1 > 1) and Next, consider the notation from Proposition 3.6, item c.Calculate If a ≤ a * , go to item c.If a ≥ a * , there is more than one equilibrium.If a * < a < a * , proceed with item b. b.Case i: Case ii: Otherwise.Let s := If a solution exists, multiple equilibria occur.If no solution exists: Solve f 2 (x) = a on I .1.If no solution exists, k i is given by ( 14) where y ∈ [ x, x r ] (see Remark 4.6.2,below) solves a = f 2 (y). 2. Precisely one solution occurs at y.If f 2 (y) = 0, or f 2 (y) = 0 and f 2 (y) > 0, k i is given by ( 14). 3. Otherwise, multiple equilibria occur.Case II.
If a unique solution y exists, go to item c.Otherwise, multiple equilibria exist.
= a has no solution on I 2 and f 2 (x) = a has no solution on I , go to item c, where y ∈ I .Otherwise, multiple equilibria occur.
Solve f 1 (x) = a on I 2 .If a solution exists, multiple equilibria occur.If no solution exists: Solve f 2 (x) = a on I .1.If no solution exists, k i is given by ( 14) where y ∈ [ x, x r ] (see Remark 4.6.2,below) solves a = f 2 (y). 2. Precisely one solution occurs at y.If f 2 (y) = 0, or f 2 (y) = 0 and f 2 (y) > 0, k i is given by ( 14). 3. Otherwise, multiple equilibria occur.c. k i is given by (13) where y solves a = f 1 (y) (see Remark 4.2.1).Remark 4.6 1.Note that, similar to the regulator game (see Remark 4.2.1),we have i=1 σ i + |a|.Again, since x r is not tight, to improve calculation speed in certain applications it might be worthwhile to improve on this bound.2. Note that, similar to the economic game, we have Again, clearly x r here is just an upperbound, which may be improved depending on the considered specific case.

An Example on Oligopolistic Competition
In this example, we consider the model on oligopolistic competition with sticky prices that was analyzed by Fershtman and Kamien [14] (see also [6]).This model describes a market where N companies sell, more or less, the same product.It is assumed that the market price does not adjust instantaneously to the price indicated by the demand function.There is a "lag" in the market price adjustment.Therefore, the price is called "sticky."Assume at any point in time, t, company i, i ∈ N, produces u i (t) with cost function and sells it in the market at the price of p.The inverse linear demand function for the product is given by Here, p, β i are positive constants. 1Next, Eq. ( 15) models that the market price does not adapt instantaneously to the price indicated by the demand function.
Here, s ∈ (0, ∞) is the adjustment speed parameter.For larger values of s, the market price adjusts quicker along the demand function.Within this framework, we look for affine stabilizing feedback Nash equilibria of the game, where every player wants to maximize his discounted profits More precisely, we assume that players choose their actions from next set F of stabilizing affine functions of the price p.
To determine the feedback Nash equilibrium actions for this game (15,16), we first reformulate it into the standard linear quadratic framework.Following [8,Example 8.5], introduce the variables Then, the problem can be rewritten as the minimization of subject to the dynamics Here According to, e.g., [8,Theorem 8.5] this game has a feedback Nash equilibrium if and only if has a set of symmetric solutions K j such that matrix A − N i=1 S i K i is stable, where where k i should be such that a − N i=1 s i k i < 0. Or, stated differently, the oligopolistic market has a noncooperative affine feedback Nash equilibrium iff (3) has a stabilizing solution with a := − 1 2 r − s(1 and q i := − 1 2 , i ∈ N. So, in the terminology of Definition 2.2, this is an economic game.Note that once we determined k i from (20), l i can be calculated from the linear Eq. (21).Finally, m i can then be directly calculated from (22).
The equilibrium actions follow now directly from (18).That is with k i given by (20).
The resulting dynamics of the equilibrium price path are Or, stated differently, with p s := In particular, we infer from this that the price in this oligopolistic market converges to p s .To find this equilibrium price for specific values of the parameters for a large number of players requires, separate from solving k i from (20), the solution of the linear Eq. ( 21).In "Appendix 2", we show how l i , i ∈ N, can be efficiently solved as (24).
To solve k i from (20), we use Algorithm 4.3.Note that So, by item 3.a of this algorithm, there is a unique solution that can be calculated as outlined in item 3.c.Table 5 reports some simulation results for the benchmark parameters c i = 1; p = 80; β i = 1; s = 2; and r = 0.1 as a function of the number of players, N .The table reports the parameters f i and g i of the affine equilibrium state feedback controls (17), together with the steady-state price p s .To see the impact if one β i parameter changes, we considered in our second experiment a case β 1 = 10 with the other parameters unchanged.Corresponding results are reported in  5.
We also performed a third experiment in which β i were chosen uniformly in the interval [0.9, 1.1].In this simulation, already for N = 10, almost the same results occurred as those reported in Table 5.
From these simulation results, we see that for a small number of players an increase in the number of players significantly affects equilibrium prices and strategies.And there is almost no impact anymore if the number of players increases above 10.Furthermore, we see from Table 6 that an increase in the β 1 parameter has more or less the same impact as increasing the number of players in the game.
For the implementation of Algorithm 4.3, we used MATLAB 2 The computation time using our code was less than 1 s using an accuracy of 10 −5 .Using a larger accuracy provided similar results.Only, when N = 10.000,problems occurred, as then apparently the numerical precision of the square root function implemented in MATLAB reached its numerical precision level.
Finally, we also implemented Algorithm 4.1 with σ i = i, i ∈ N for this example and were able to calculate equilibria up to N = 100.000,within 1 s.But, also in this case, numerical problems seemed to occur for a larger number of players.

Concluding Remarks
In this paper, we presented numerical algorithms to determine the unique linear feedback Nash policies of the basic scalar linear quadratic differential game.We distinguished three cases: the regulator, the economic, and the mixed game.For each game, we provided a separate algorithm which calculates the unique equilibrium if it exists.Furthermore, the algorithm indicates for the economic game, in case not a unique equilibrium exists, whether no equilibrium or multiple equilibria occur.
See item 1 Regulator game.

Symmetric case
Not applicable.3. General case a.This part of the proof is identical to the proof of the corresponding part of the regulator game.b.Case i.This case follows along the lines of the proof of the regulator game.
Next note that f 1 (x) = h 1 (x) + f e 1 (x), where h 1 (x) = It is easily verified that h 1 (x) < 0, ∀x ≥ √ σ 1 .Furthermore, similar as in the proof of Algorithm 4.3 item 3b, it follows that f e 1 (s) < 0 and, consequently, f e 1 (x) < 0, ∀x ≥ s.So f 1 (x) < 0, ∀x ≥ s.Since, moreover, f 1 (x) < f 3 (x), it follows that f 1 can only have (at most two) stationary points on I 2 .Case I.i: Since f 1 (s) ≥ f 1 ( x), it follows that for all a < f 1 ( x), the equation f 1 (x) = a always has a solution y > s.From this, the result follows immediately.Case I.ii: Similar as above, it follows that f 1 (x) = a always has a solution y > s if a ∈ ( f 1 ( x), f 1 (s)].Furthermore, additionally, for x < s always either f 1 (x) = a or f 2 (x) = a has a solution.Case I.iii: If a ∈ [ f 1 (s), f 3 * ] always, the equation f 2 (x) = a will have a solution.So, if f 1 (x) = a has at least one solution multiple equilibria exist.Note that on I , f 1 (x) ≤ f 1 ( √ σ 1 ) and for x ≥ s, f 1 (x) ≤ f 1 (s).So, the search for solutions of f 1 (x) = a can be restricted to I 2 .In case no solution exists, equilibria are obtained from the intersection points of f 2 (x) = a.Since f 2 (x) > f 3 (x), it follows that f 2 is strictly increasing for x ≥ x.Therefore, in case f 2 (x) = a has no solution on I there will be a unique solution attained at the intersection point of f 2 (x) = a outside I .In case there is a unique solution y on I , this yields the unique equilibrium, unless f 2 attains a local maximum at y. From these observations, the stated results follow.Case II.iIn this case, the equation f 1 (x) = a always has a solution y > s.From this, the result follows immediately.
Case II.iiNote that f 2 is monotonically increasing for x ≥ x and f 2 ( x) > f 1 ( x).So, f 2 (x) = a has no solution for these values of a.All equilibria are obtained from the intersection points f 1 (x) = a in this case which yields the stated result.
Case II.iiiIn this case, f 1 (x) = a has a unique solution on I .So if either f 1 (x) = a has a solution on I 2 or f 2 (x) = a has a solution on I (notice again, f 2 is monotonically increasing for x ≥ x), multiple equilibria occur.
Case II.ivThis case is similar to Case I.iii.

Table 3
Number of equilibria if σ

Table 4
Number of equilibria, general case (10)ulate, with t 1 = 1, S and y from(10).If either a < f 1 (0) or a ∈ ( f 2 (0), f 3 (0)) holds, proceed with item c.Otherwise, proceed with item b. b.Calculate f * 1 (see Table the solution of f 2 (y) = a in item 3.b.II.ii can be obtained, e.g., by calculating the unique zero of F 2 (x) on the interval [ x, x r ].

Table 5
Equilibrium actions in case βi = 1

Table 6 .
In this table, no results are reported for N > 100 because they coincide with those of Table