Secure Discrete-Time Linear-Quadratic Mean-Field Games

Abstract

In this paper, we propose a framework for strategic interaction among a large population of agents. The agents are linear stochastic control systems having a communication channel between the sensor and the controller for each agent. The strategic interaction is modeled as a Secure Linear-Quadratic Mean-Field Game (SLQ-MFG), within a consensus framework, where the communication channel is noiseless, but, is susceptible to eavesdropping by adversaries. For the purposes of security, the sensor shares only a sketch of the states using a private key. The controller for each agent has the knowledge of the private key, and has fast access to the sketches of states from the sensor. We propose a secure communication mechanism between the sensor and controller, and a state reconstruction procedure using multi-rate sensor output sampling at the controller. We establish that the state reconstruction is noisy, and hence the Mean-Field Equilibrium (MFE) of the SLQ-MFG does not exist in the class of linear controllers. We introduce the notion of an approximate MFE ($\epsilon$-MFE) and prove that the MFE of the standard (non-secure) LQ-MFG is an $\epsilon$-MFE of the SLQ-MFG. Also, we show that $\epsilon \rightarrow 0$ as the estimation error in state reconstruction approaches 0. Furthermore, we show that the MFE of LQ-MFG is also an $(\epsilon +\varepsilon )$-Nash equilibrium for the finite population version of the SLQ-MFG; and $(\epsilon +\varepsilon ) \rightarrow 0$ as the estimation error approaches 0 and the number of agents $n \rightarrow \infty$. We empirically investigate the performance sensitivity of the $(\epsilon +\varepsilon )$-Nash equilibrium to perturbations in sampling rate, model parameters, and private keys.

Research support in part by Grant FA9550-19-1-0353 from AFOSR, and in part by US Army Research Laboratory (ARL) Cooperative Agreement W911NF-17-2-0196.

7 Appendix

In this section, we provide some necessary background material for completeness.

1.1 7.1 MFE of the LQ-MFG

Here we briefly discuss the MFE of the LQ-MFG which has been developed in a previous work [18]. We note from [18] that the dynamics of the generic agent in the LQ-MFG are given by,

$$\begin{aligned} x((k+1)\tau ) = A_0 x(k \tau ) + B_0 u (k \tau ) + w^0 (k \tau ) \end{aligned}$$

(24)

Although $w^0$ is assumed to be non-Gaussian (Sect. 2.2) the results of [18] (which assume Gaussian distribution) still hold, since we restrict our attention to the class of linear controllers. In the standard LQ-MFG, the multi-rate setup is not required since the controller has access to the true state of the agent. The generic agent aims to minimize the cost function,

$$\begin{aligned} \tilde{J}(\mu ,\bar{x}) = \limsup _{T \rightarrow \infty } \frac{1}{T} \mathbb {E}_{\mu } \Big \{ \sum _{k=0}^{T-1} ||x(k \tau ) - \bar{x}(k \tau ) ||_{Q}^2 + ||u(k \tau )||_R^2 \Big \}, \end{aligned}$$

(25)

where $\bar{x}$ is the mean-field trajectory. Next we restate the existence and uniqueness guarantees of MFE for the LQ-MFG.

Proposition 1

([18]). Under Assumption 1 the LQ-MFG ((24)–(25)) admits the unique MFE given by the tuple $(K^*,F^*) \in \mathbb {R}^{p \times 2m} \times \mathbb {R}^{m \times m}$. The matrix $F^* = \varLambda (K^*) = A_0 - B_0(K^*_1 + K^*_2)$, and controller $K^*$ is defined as,

$$\begin{aligned} K^* = (\bar{B}^T P^* \bar{B} + R )^{-1} \bar{B}^T P^* \bar{A}^*, \text { where } \bar{A}^* = \begin{bmatrix} A_0 &{} 0 \\ 0 &{} F^* \end{bmatrix} \end{aligned}$$

(26)

and $P^*$ is the solution to the DARE,

$$\begin{aligned} P^* = \bar{A^*}^T P^* \bar{A^*} + \bar{Q} - \bar{A^*}^T P^* \bar{B}(R + \bar{B}^T P^* \bar{B})^{-1} \bar{B}^T P^* \bar{A^*} \end{aligned}$$

(27)

and $\bar{B}$ and $\bar{Q}$ as defined in (11) and (12), respectively.

The DARE is obtained by substituting $K^*$ in the Lyapunov Eq. (16) hence $P^* = P_{K^*}$. An important point to note is that in the LQ-MFG the estimation error is 0, as the controller has perfect access to the state of the agent. This translates to the covariance matrix of estimation error $\varSigma _C = 0$ and hence $\hat{\varSigma }_C = 0$ for LQ-MFG. Using (15) the cost of linear controller K and linear trajectory defined by matrix F for the LQ-MFG will be

$$\begin{aligned} \tilde{J}(K,F) = \mathrm{Tr}\,(P_K \bar{\varSigma }) \end{aligned}$$

(28)

where $P_K$ is the solution to the Lyapunov Eq. (15). Furthermore it can also be verified that

$$\begin{aligned} K^* = \mathrm{argmin}_K \tilde{J} (K, F^*) \end{aligned}$$

(29)

This MFE $(K^*,F^*)$ can be obtained by using the mean-field update operator as discussed in [18].