Distributed constrained aggregative games of uncertain Euler-Lagrange systems under unbalanced digraphs

In this paper, the constrained Nash equilibrium seeking problem of aggregative games is investigated for uncertain nonlinear Euler-Lagrange (EL) systems under unbalanced digraphs, where the cost function for each agent depends on its own decision variable and the aggregate of all other decisions. By embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the digraph Laplacian matrix, a dynamic adaptive average consensus protocol is employed to estimate the aggregate function in the unbalanced case. To solve the constrained Nash equilibrium seeking problem, an integrated distributed protocol based on output-constrained nonlinear control and projected dynamics is proposed for uncertain EL players to reach the Nash equilibrium. The convergence analysis is established by using variational inequality technique and Lyapunov stability analysis. Finally, a numerical example in electricity market is provided to validate the effectiveness of the proposed method.


Introduction
With an effective integration of sensing, communication, computation, optimization and decision making with physical dynamics, flourishing research activities on cyberphysical systems have been observed in the last decade. In particular, the distributed strategies combined with physical dynamics have drawn increasing research attention in resource allocation [1], smart grid [2,3], mobile robot network [4], and autonomous vehicles coordination [5][6][7]. Due to the flexibility and unification in the modeling, EL systems as one typical class of physical systems have been extensively investigated to describe many mechanical systems, such as manipulator robots, rigid bodies, underwater vehicles and spacecrafts (see [8,9] and references therein). Leaderless and leader-following consensus problems for EL systems were investigated in [10][11][12], dis-ables, but also related to an unknown aggregate function defined by decision variables of all players, an example being the well-known Cournot model where the aggregate supply of opponents that matters rather than their individual strategies(see [18] and the references therein). Therefore, in aggregative game algorithms, all players only need to estimate the aggregate terms rather than all the decision variables, which can reduce the complexity and computation cost of the game playing algorithm on one hand; on the other hand, for each player, the influence of uncertain factors in the game environment on the estimated value of aggregate term may be less than the total influence on the estimated values of all individual decision variables, which can improve the predictability of aggregate games to some extent. As one of the effective game model, aggregation game have been widely utilized to predict and analysis strategies in many economic and social fields [14,16,18,19], such as the prediction and analysis of public resources, market prices, elections and voting, etc.
Although remarkable achievements have been made on this topic during the past few years, most of the existing studies focus on the first-order integrator system [14,20,21], and the complicated dynamic of physical systems may have an effect on the implementation of game strategies in practical applications [3,4]. There are still many problems involving with the uncertainties, directed or switching topology, and constraints for the complicated highorder nonlinear dynamics. However, relatively few results have been obtained for these promising topics, especially for the integration of these cases mentioned above. In [4], Deng made the first step to solve the unconstrained aggregative games of EL systems, and then extended the results to the equality constraint case in [3]. The unconstrained aggregative game for high-order nonlinear dynamics with the external disturbance was investigated in [7]. However, the communication graphs among most of the existing literatures were assumed to be undirected [3,4,7,14,21] or weight-balanced directed [22,23]. Additionally, some prior knowledge of the global parameters is required in the implementation of most of the above algorithms, such as the Laplacian matrix [3,4]. Furthermore, nearly all practical physical systems are subject to output constraints coming from specifications of operation and considerations of safety [24]. However, most of the existing results are about the unconstrained game or equality constraint game [3,4,7,23].
Motivated by the above discussion, in this paper, we consider the distributed constrained aggregative game for uncertain El systems under unbalanced directed communication topology. The main contributions are summarized as follows: (i) Different from most of the existing literatures about the distributed aggregative games of the first-order integrator systems [14,20,21], the distributed optimization of EL systems [5], and the distributed unconstrained aggregative game of EL systems [3,4], we consider the distributed aggregative game of uncertain EL systems with set constraints. This problem is more challenging under the concurrent influence of the uncertain nonlinear EL dynamics and the output constraints. Motivated by the outputconstrained control design techniques in [25], we propose a fault compensation scheme to conquer the above difficulties, which is totally different from the tracking controller design in [3][4][5].
(ii) The communication topology among agents is described by a weight-unbalanced digraph, which is different from most of the existing results [3,4,7,14,[21][22][23]. In order to tackle the difficulties caused by unbalanced communication topology, we design a distributed estimator to estimate the left eigenvector of the Laplacian matrix associated with the zero eigenvalue.
(iii) Different from [3,4], we proposed a fully distributed protocol for EL systems to solve the constrained aggregative games under unbalanced digraphs, and the selection of control parameters does not depend on any global information.
This paper is organized as follows. In Sect. 2, we formulate the considered problem along with basic concepts. In Sect. 3, we present a distributed algorithm for EL systems to reach the Nash equilibrium of aggregative games with a prescribed accuracy, and then we give the main results and convergence analysis under unbalanced directed interaction topologies. Following that, we carry out simulation studies to verify our theoretical results in Sect. 4. Finally, we give concluding remarks in Sect. 5.
Notations: · and · ∞ denote the Euclidean norm and L ∞ of a vector or matrix, respectively. For x i ∈ R m , i = 1, 2, . . . , N , x j i is the j-th element of x i . Let 1 n and 0 n be the n-dimensional vectors of all entries as 1 and 0, respectively.

Problem formulation
In this section, we first review some related preliminaries in graph theory [26], convex analysis [27] and variational inequality [28,29].

Preliminaries
. . . , N} is the node set, E ⊆ V × V is the edge set, and A = [a ij ] ∈ R N×N is the weighted adjacency matrix. If (i, j) ∈ E, then a ij > 0, which means that the agent i can receive information from agent j; otherwise, a ij = 0. Denote N i = {j ∈ V|(i, j) ∈ E} as the set of neighbors of agent i. The degree matrix of the graph G is defined as A directed graph G is strongly connected if there exists a directed path between any two nodes in V. If G is strongly connected, then zero is the simple eigenvalue of L, and all the other eigenvalues have positive real parts.
A set K ⊆ R m is convex if ax+(1-a)y ∈ K for any x, y ∈ K and a ∈ [0, 1]. For any x ∈ R m , there is a unique element P K (x) ∈ K satisfying x -P K (x) = inf y∈K xy , which is denoted by |x| K , where K ⊆ R m is a nonempty closed convex set and P K denotes the projection operator onto K . Denote |x| K = 0 for any x if K = ∅, for convenience. Denote N K (x) = {d| d, yx ≤ 0, ∀y ∈ K} as the normal cone of K at x, and T K (x) = {lim n→∞ x n -x t n |x n ∈ K, t n > 0, x n → x, and t n → 0} as its tangent cone. The following are some useful results given in [30,31] about the properties of projection operator.

Lemma 1 Let K be a closed convex set in
Let K ⊂ R m be a closed convex set, and F(x) : K → R m is a map, then the solution of variational inequality can be equivalently reformulated via the following projection where VI(K, F) represents the variational inequality (1) for convenience, and SOL(K, F) denotes the set of its solutions.

Problem statement
Consider a constrained aggregative game of N heterogeneous nonlinear EL players (agents) with an associated directed communication graph G = (V, E). The dynamics of each player i ∈ V is described as follows: where q i ,q i ∈ R m denote the generalized position and velocity vectors, respectively; M i (q i ) ∈ R m×m is the positive definite inertia matrix; C i (q i ,q i )q i ∈ R m is the vector for Coriolis and centripetal forces; G i (q i ) ∈ R m is the gravity vector; and τ i ∈ R m is the control force. Note that q i ,q i are measurable, and M i (q i ), C i (q i ,q i ), G i (q i ) are permitted to be unknown. The dynamics of the system (3) satisfies the following property, which was widely used in [25,32].
In these EL systems, player i is endowed with a local pay- i is the feasible strategy set of player i, which is driven by the fact that most of mechanical systems are subject to physical constraints for safe operation. The local payoff function J i (q i , q -i ) is only known by player i, and it is not only dependent on its own decision variable q i , but also related to the aggregate of the decision variables of all players. We define the aggregate function σ : → R n as follows: ) with a function i : R m+n → R. Note that σ (q) is unknown to all the players, and each player i can only obtain the partial information φ i (q i ) of the aggregate σ (q). Moreover, players can not share their payoff functions and the local contribution functions to others for privacy.
The objective of player i is to minimize its own payoff function J i (q i , q -i ) by choosing the appropriate decision in its safe output constraint q i ∈ i , which is equivalent to solve the following optimization problem: Then we introduce the concept of Nash equilibrium (NE) and some related results (see [21] for details).
which means NE is a profile on which no player can further decrease its local payoff function by changing its own decision unilaterally.
Define a map F : R mN → R mN as a strategy profile is called a variational NE if it is a solution of VI( , F). Moreover, Theorem 3.9 of [33] shows that if is a convex, every solution of VI( , F) is also an NE.
There are some assumptions about the communication graph and the variational NE.

Assumption 1
The communication graph G is directed and strongly connected. Under Assumption 2 and 3, it is not hard to obtain the following results with a similar analysis in [21].

Lemma 3 Suppose Assumption 2 and 3 hold, the constrained aggregative game (4) has a unique variational NE q * ∈ . Moreover, q * is a variational NE if and only if
Proof According to Theorem 2.2.3 and Corollary 2.2.5 in [28], the existence of a variational NE can be guaranteed by Assumption 3 and the continuous differentiability of J i (q i , q -i ) in Assumption 2; and the uniqueness can be guaranteed by the strongly monotonicity of F(q) in Assumption 2.
Next, we formulate our considered problem. (3) to seek the variational NE of the constrained aggregative game (4) under unbalanced digraph such that • the EL system outputs reach the variational NE with prescribed accuracy, i.e.,

Problem 4 Design a distributed protocol for uncertain nonlinear EL systems
where is a preselected arbitrary positive constant; • the specified output constraint q i ∈ i is never transgressed; • all signals in the closed-loop system are bounded.
Remark 1 Different from the well-studied distributed aggregative games in [14,20,21], our formulation gets involved with the nonlinear EL dynamics of each player. The considered formulation has wide potential applications in many distributed cyber-physical systems which involve the dynamics of both scheduling system and operators, such as the demand response management of power systems [14], electricity market games of smart grids [3,4], routing planing of electric vehicles [17], multirobot basketball game [34], multi-robot surveillance application [35] and other fields. Note that the intrinsic nonlinear and uncertain dynamics of EL players make the constrained NE seeking problem of EL systems much more challenging than the linear integrator case. Additionally, in contrast to most of the existing NE seeking algorithms under the undirected graph in [3,4,21] or weight-balanced digraph in [22,23], the communication network in this paper is unbalanced, which indicates that the considered problem enables more extensive scope of application. Furthermore, the above mentioned works did not consider the constraints on actuators, which may deter the implementation on physical systems. In our controller design, all signals in the closed-loop system are required to be bounded for safe and reliable practical implementation.

Main results
In this section, we propose a distributed protocol for uncertain EL systems (3) to solve the variational NE seeking Problem 4.

Algorithm design
In practical situations, the system parameters may be unavailable for each player, together with the output constraints, which makes the feedback linearization method and the path tracking method in [3][4][5] fail to deal with the constrained and uncertain nonlinear EL system (3). Motivated by the fault compensation technique in [25], a distributed output-constrained controller is proposed for unknown EL systems with output constraints.
The distributed controller of player i, i ∈ V, which consists of three parts, is designed as follows: (1) the local output-constrained controller (2) the cooperative game optimizeṙ where The virtual variable x is used to estimate the variational NE q * , and the auxiliary variables s i , y i ∈ R n are used to estimate the aggregate function σ (x) = 1 where ii is the i-th element of i .˜ can be termed as the estimator of the left eigenvector of L associated with the zero eigenvalue.
where i (0) = col(0 i-1 , 1, 0 N-i ). Figure 1 depicts the main framework of the above algorithm. The estimator (9) is used to estimate the left eigenvector associated with zero eigenvalue of the unbalanced Laplacian matrix, which can be used to eliminate the influence of unbalance on the cooperative game optimizer (8a)-(8c); (8a)-(8c) is used to seek the variational NE and generates the reference path; and the local output-constrained controller (7a)-(7h) aims to force the EL system to track the reference path in a safe set constraint.
This way of set contraction could effectively guarantee the trajectories of practical EL system never transgress the boundaries of i . Meanwhile, the variational NE seeking accuracy can be improved by reducing the value of . Additionally, if the variational NE q * of aggregative game (4) is an interior point, we can conclude that the slight scaling of the boundaries does not affect the NE of (4). Furthermore, our proposed dynamic adaptive average consensus algorithm (8b), (8c) is totally different from those given in [3,4,36]. We design a distributed estimator to eliminate the influence of unbalance on (8b), (8c).

Convergence analysis
Denote ω = (ω 1 , . . . , ω N ) T as the left eigenvector of L associated with the zero eigenvalue, i.e., ω T L = 0. Without loss of generality, we assume ω T 1 = 1. Then we can obtain the following lemma.

Theorem 7 Suppose Assumptions 1-3 hold. If control parameters satisfy
Proof First, we show that the trajectories generated by the cooperative game optimizer (8a)-(8c) converge to the variational NE q * = col(q * 1 , . . . , q * N ) of (4), i.e., Notice that, in (8a), if x i ∈ K i holds, we can conclude thatẋ i ∈ T K i (x i ). According to the Nagumo's theorem in Page 7 of 14 [30], we obtain x i (t) ∈ K i , ∀t ≥ 0 under the initial condition σ (x))), it results from Lemma 1 and Assumption 2 that Together with Lemma 6, we conclude that e i exponentially converges to 0. With the above change of variable, we can rewrite (8a) as followṡ which can also be described in a compact form aṡ Define H(x) = P K (x -F(x)), and then it results from [29] that Construct the following Lyapunov function then differentiating it along (20) leads tȯ With the boundedness of x i ∈ K i and the exponential convergence of e, we conclude that The first term inV can be decomposed as    H(x)) ≥ 0. By using the first property in Lemma 1, we obtain ((x -F(x)) -P K (x -F(x))) T (q * -H(x)) ≤ 0. Furthermore, it results from (6) that -F(q * ) ∈ N K (q * ), i.e., -F T (q * )(xq * ) ≤ 0. Thus, we havė With the strong monotonicity and the Lipschitz continuity of F(x), we can get that F(x) is uni-   (13) in [4] formly continuous. And (F(x) -F(q * )) T (xq * ) = 0 if and only if x = q * . Together with the Barbalat' lemma in [38], we have lim t→∞ x(t) = q * .
Next, we prove that the reference trajectories x i , ∀i ∈ V can be tracked by the practical EL systems with a prescribed accuracy , i.e., First, we need to prove that both the state error and velocity error are preserved in their respective constraint sets Suppose by contradiction that there exists t 1 > 0 such that the violation of the above inequalities occurs firstly. Since |z where . Since x i ∈ K i , it results from (8a) thatẋ j i is bounded. Then it is not hard to get the From (26) and (27),V j i < 0 as |ln As a result, Together with the injective property of the nature logarithm function, it is essential that ς j i can not approach k j i and k j i for any t < t 1 .

Consider the following Lyapunov function
where ). It is not hard to derive the boundedness of F i , which means that there exists a positive constantλ i such that It results from the positive definiteness of M i that there exists a positive constantλ i such that It results from (7b) and (7c) that there existsξ i such that Together with (30) and (31) thaṫ . As a result, we obtain that V i (t) ≤ Due to the injective property of the tangent function (7c), z j i can not approach ξ j i , which means ( i ) -1 is bounded. Therefore, the supposition that the state error and velocity error may transgress the prespecified boundary is impossible, and the expected result (24) holds. It follows from (7f) and (7g) that Additionally, with the boundedness of η i and ( i ) -1 , we can conclude that the control input τ i is bounded. The proof is completed.

Simulations
In this section, we give numerical simulations to illustrate the convergence performance of the proposed distributed algorithm in this paper.
Consider an energy resource competition problem among N generating firms in electricity market(see [3,4] and references therein), which can be formulated by the following aggregative game: where q i is the output power of generation system i, i = [ν i ,ν i ] ⊂ R is the local constraint set withν i being the local load demand andν i being the maximum allowed output power. J i is the payoff function of generation system i, which is defined as where a i + β i q i + r i q 2 i is the generation cost with a i , β i , r i being the characteristics of generation system i, and p 0 -aNσ (q) is the electricity price with constants p 0 , a and the aggregate term σ (q) = N i=1 q i N describing the average output power.
The dynamics of turbine-generator i can be described as follows [39]: where d i is the relative speed of system i; d 0 is the synchronous machine speed; R i is the regulation constant of turbine; K mi is the gain of turbine i; T mi and T ei are the time constants of the ith turbine and speed governor, respectively; D i and H i are the positive per unit damping constant and inertia constant,respectively; τ i is the control input of the ith generation system. The zero-system of (33a) is which can be rewritten as the standard EL equation (3) with M i = T mi T ei K mi , C i = T mi +T ei K mi and G i = 1 K mi q i . It follows from Theorem 7 that the zero-system (34) exponentially converges to the variational NE of (4) under the proposed algorithm (7a)-(7h), (8a)-(8c), (9). Since D i and H i are positive, d i vanishes exponentially, which implies that the generator system (33a)-(33b) also exponentially converges to the same variational NE via algorithm (7a)-(7h), (8a)-(8c), (9). Therefore, the considered electricity market game problem can be solved by our formulation and al- Take the aggregative game with 5 generation systems under an unbalanced digraph, whose communication topol-ogy is described in Fig. 2, and the weighted adjacency matrix A = [a ij ] is given by a ij = 1, if (i, j) ∈ E.
Taking p 0 = 25, a = 0.1, (a 1 , . . . , a 5 ) = (5,8,6,9,7), (β 1 , . . . , β 5 ) = (12,10,11,11,13), (r 1 , . . . , r 5 ) = (1, 5, 0.8, 2, 1.1), i = [-10, 10], K i = [-0.99, 0.99], ∀i = 1, . . . , 5, d 0 = 314.159. The variational NE of aggregative game is q * = (5.2036, 1.2799, 7.0162, 2.9091, 4.3163). The generator system and controller parameters are given in Table 1. The initial values are given as s i (0) = y i (0) =q i (0) = 0, α i = 0.005, β = 0.2, Firstly, let us study the convergence performance of our proposed algorithm compared with the distributed Nash equilibrium algorithm (13) in [4]. As show in Fig. 3 and Fig. 4, the output power trajectories of five generator systems converge precisely to the variational EN q * of the considered aggregative game under the unbalanced digraph Fig. 2 via the proposed algorithm, and all of the velocity trajectories converge to the zero. This is because q * is the inner point of the constraint set , and the slight scaling of the boundaries does not affect the Nash equilibrium of the considered problem. However, it can be seen from Fig. 5 that the distributed Nash equilibrium algorithm (13) in [4] can not guarantee the output power trajectories of five generator systems converge to the variational EN q * of the considered problem, and some of the output power trajectories are outside the safety constraints. Therefore, the algorithm (13) in [4] is not suitable for solving set constrained games and non-balance situations. In contrast, an integrated distributed protocol based on fault compensation scheme and projected dynamics is proposed in this paper to conquer the difficulties caused by set constraints. Figure 6 gives the evolutions of control torques of five EL systems under the proposed protocol, showing that the control torque τ i of each system is bounded. Moreover, from Figs. 7 and 8, it can be observed that all the trajectories of EL systems converge to the optimal reference path generated by the cooperative game optimizer under the local output-constrained controller (7a)-(7h), and the traking errors converge to 0 along with the imposed boundaries.
Next, we randomly choose another strongly connected communication graph Fig. 9 and select the same initialization conditions with Figs. 3 and 4. It can be seen from Figs. 10 and 11 that the output power trajectories of five EL systems also converge precisely to the variational EN q * under the proposed algorithm, supporting the theoretical result that our proposed algorithm can handle the unbanlanced strongly connected case.
To show the influence of the initial states on the convergence of the proposed algorithm, we randomly generate six different initial values, and Fig. 12 gives the evolutions of output powers in different cases, which implies that for any initial states q i (0) ∈ i , their values do not affect the convergence of the proposed algorithm. Figures 13 and 14 show the evolutions of output power trajectories and control torques under different control gains γ i = 0.01, 0.1, 1, 10, 20, 30, respectively. From Fig. 14, we can observe that the control torques are bounded, and the size of the bound will decrease as the control gain increases, which implies that the proposed algorithm is also applicable to physical systems with control input constraints or limited actuators. Therefore, the above simulation results verify the effectiveness of our proposed protocol and support the theoretical results.

Conclusions
In this paper, we considered the distributed aggregative games for multiple heterogeneous EL systems under unbalanced digraph. We proposed a distributed protocol for uncertain nonlinear EL systems to seek the variational GNE based on the output-constrained control, projection algorithm and dynamic average consensus techniques, and gave the corresponding convergence results. Simulation results verified the effectiveness of our proposed protocol. Future works include the aggregative game for high-order nonlinear multi-agent dynamics with constraints.

Funding
This work is supported by National Natural Science Foundation of China under Grants 61703368, 62073107, and Natural Science Foundation of Zhejiang Province under Grants LZ21F030002.