Abstract
In this paper, a distributed stochastic approximation algorithm is proposed to track the dynamic root of a sum of time-varying regression functions over a network. Each agent updates its estimate by using the local observation, the dynamic information of the global root, and information received from its neighbors. Compared with similar works in optimization area, we allow the observation to be noise-corrupted, and the noise condition is much weaker. Furthermore, instead of the upper bound of the estimate error, we present the asymptotic convergence result of the algorithm. The consensus and convergence of the estimates are established. Finally, the algorithm is applied to a distributed target tracking problem and the numerical example is presented to demonstrate the performance of the algorithm.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In recent years, there is a huge increase in the scale of data in many real-life application problems. The traditional centralized computing method faces many challenges and sometimes is entirely infeasible for large-scale problems. As a result, distributed algorithms over multi-agent systems have received much attention from researchers of diverse areas, including consensus problem [1–4], resource allocation (RA) [5, 6], multi-unmanned aerial vehicle (MUAV) control [7], and distributed target tracking [8] etc. Distributed algorithms are usually associated with a network of agents where each agent has limited computation and communication ability. The agents are required to cooperatively achieve a global objective by using their local observations and information transmitted from their neighbors. Compared with centralized approaches, distributed algorithms have the advantage of robustness on network link failure, privacy protection, and reduction on communication and computation cost.
One of the important branches of distributed algorithms is the distributed optimization problem. It seeks the minimizer of the global function which is written as a sum of the local functions of the agents. In particular, the distributed optimization for time-invariant cost function has become a mature discipline with many results, see [9–11] and references within. On the other hand, optimization problems with time-varying cost function have attracted much attention due to its appearance in various applications, for example, signal processing [12] and online optimization [13, 14]. The main challenge of time-varying optimization lies in the fact that the minimizer of the time-varying cost function is changing with time. Since the traditional optimization algorithms can only move the estimates towards the minimizer of the cost function of the current time, they cannot track the movement of the minimizer in the dynamic environment. To cope with this issue, two different strategies have been developed. The first one is running method [15, 16], where the algorithms sample the time-varying cost function at a fixed frequency, and perform the traditional optimization algorithms on the sampled function in between the sample time. The second one is prediction-correction method [17, 18], where the algorithms only optimize the cost function of the current time at each steps. But an additional information of the dynamic of the moving minimizer is required, such as the second derivative of the cost function [18], or an additional constrain on the dynamic of the minimizer [14]. While we consider a different problem in this paper, the method we utilize is much similar to the second method.
It can be noticed that the optimization problem can often be transformed into a root-seeking problem, since the optimization of a differentiable convex function is equivalent to seeking the root of its gradient, and for the case where the gradients are unavailable we can use the finite time difference to estimate the gradient [19]. So it’s natural to consider the distributed root-tracking problem. Distributed stochastic approximation for time-invariant regression functions has been studied by many researchers as a solution for distributed root-seeking problem [20–22]. Inspired by the work of dynamic stochastic approximation [23, 24], in this paper, we propose a distributed stochastic approximation algorithm for tracking the changing root of a sum of time-varying regression functions over a network. Each agent is aimed at tracking the changing root of the global function, but it can only access a noise-corrupted local observation and the information transmitted from its neighbor. In addition, the noise-corrupted dynamics of the roots of the global regression function is assumed to be known to all agents.
In this paper, the distributed root-tracking problem for time-varying regression function is considered. First, motivated by the truncation technique given in [22], a distributed stochastic approximation algorithm with expanding truncations is introduced. The key difference is that the observation of the local function in this algorithm is a noise-corrupted one, while an exact gradient information is often required in the optimization algorithms mentioned above. Second, under the assumption that the noise-corrupted dynamics of the global roots is known to all agents, the convergence conditions of the algorithm are introduced. Third, it is proved that the estimates generated by the distributed algorithm are of both consensus and convergence with probability one. Finally, we apply this algorithm to a distributed target tracking problem. The numerical example is given demonstrating the performance of the algorithm.
The rest of the paper is organized as follows. The problem formulation and the distributed stochastic approximation algorithm are given in Section 2. The convergence conditions and results are presented in Section 3. To help the proof of the convergence result, two auxiliary sequences are defined and analysed in Section 4. The proof of the main result is given in Section 5. In Section 6, a distributed target tracking problem is solved by the algorithm and the numerical example is demonstrated. Some concluding remarks are addressed in Section 7.
2 Problem formulation and distributed root-tracking algorithm
2.1 Problem formulation
Consider a network system consisting of N agents. The interaction relationship among agents is described by a time-varying digraph \(\mathcal {G}(k)=\left \{\mathcal {V},\mathcal {E}(k)\right \}\), where k is the time index, \(\mathcal {V}=\{1,\dots,N\}\) is the agent set, and \(\mathcal {E}(k)\subset \mathcal {V}\times \mathcal {V}\) is the edge set. By \((i,j)\in \mathcal {E}(k)\) we mean that agent j can receive information from agent i at time k. Assume \((i,i)\in \mathcal {E}(k)\) for \(\forall k=1,2,\dots \) Denote the neighbor of agent i at time k by \(N_{i}(k)=\left \{j\in \mathcal {V}:(j,i)\in \mathcal {E}(k)\right \}\). The adjacency matrix associated with the graph is denoted by \(W(k)=\left [w_{ij}(k)\right ]_{i,j=1}^{N}\), where wij(k)>0 if and only if \((j,i)\in \mathcal {E}(k)\), otherwise wij(k)=0.
A time-independent digraph \(\mathcal {G}=\{\mathcal {V},\mathcal {E}\}\) is called strongly connected if for any \(i,j\in \mathcal {V}\), there exists a directed path from i to j. A directed path is a sequence of edges \(\left (i,i_{1}\right),\left (i_{1},i_{2}\right),\dots,\left (i_{p-1},j\right)\) in the digraph with distinct agents \(i_{k}\in \mathcal {V},~0\le k\le p-1\), where p is called the length of this path. A nonnegative matrix A is called doubly stochastic if A1=1 and 1TA=1T.
The time-varying global regression function is given by
where \(f_{i,k}(\cdot):\mathbb {R}^{l}\to \mathbb {R}^{l}\) is the local function associated with agent i. Denote by θk the root of the sum function fk(·) at time k, i.e., \(f_{k}(\theta _{k})=0,~ k=1,2,\dots \)
Further, assume that the dynamics of the root θk is governed by
where the function \(g_{k}(\cdot):\mathbb {R}^{l}\to \mathbb {R}^{l}\) is known for all agents, and {ξk} is the sequence of dynamic noises. As we can see in Section 6, this assumption is reasonable in some real-life application problems and have been studied before in [14, 25, 26].
For each agent i, the distributed root-tracking problem is to track the dynamic root of the time-varying global function by using its noise-corrupted observation of local function fi,k(·), the dynamic information of the root gk(·), and the information obtained from its adjacent neighbors.
2.2 Algorithm
We now introduce the distributed root-tracking algorithm as follows:
where 1) \(x_{i,k}\in \mathbb {R}^{l}\) is the estimate of θk given by the agent i at time k, 2) Oi,k+1 defined by (7) is the local observation of agent i, 3) {ak}k≥0 is the sequence of the step-sizes used by all agents, 4) x∗ is a fixed vector in \(\mathbb {R}^{l}\) known to all agents, 5) {Mk}k≥0 is a sequence of positive numbers increasingly diverging to infinity with M0≥||x∗||, 6) σi,k is the truncation number of agent i up-to-time k, and 7) hk(·) is a function defined as below
Let us explain the algorithm. 1) For agent i, the estimate xi,k is the estimate of θk. Since the dynamics of {θk} is governed by (2), in order to make sure the estimate track the dynamic root, the update at time k+1 utilize gk(xi,k) instead of xi,k as it was shown in (3). 2) For agent i, the truncation happens when one of the following cases hold true: a) \(\sigma _{i,k}<\hat {\sigma }_{i,k}\), which means that there is at least one neighbor whose truncation number is larger than that of agent i. b) \(||x^{\prime }_{i,k+1}-h_{k}(x^{*})||> M_{\hat {\sigma }_{i,k}}\), which means that the distance between the intermediate value \(x^{\prime }_{i,k+1}\) and hk(x∗) is larger than the truncation bound. When truncation happens, the estimate xi,k is pulled to hk−1(x∗). 3) It can be seen that the truncation may not happen at the same time for different agents in the network. So for agent i, the update (5) makes sure that the truncation number of i is not smaller than the largest truncation number of its neighbors, i.e. \(\hat {\sigma }_{i,k}\). As to be shown in Lemma 4, this technique guarantees that the difference between truncation numbers of different agents is bounded, which helps the algorithm converge. 4) The truncation mechanism makes sure that the estimates xi,k won’t be too far away from hk−1(x∗). As to be shown in Lemma 1, we can prove that the distance between {hk−1(x∗)} and the dynamic root {θk} is bounded. So it is reasonable to choose this truncation condition.
Remark 1
In our previous work [27], we proposed the distributed root-tracking algorithm without the expanding truncation. To make sure the algorithm converge, we assumed the dynamic root {θk} and the estimate of all agents {xi,k} are bounded sequences in [27]. With the introduction of the expanding truncation mechanism, this assumption is removed in this paper.
3 Assumptions and convergence result
3.1 Assumptions
Let us list the assumptions to be used in the paper.
-
\(a_{k}>0,a_{k}\to 0,\sum _{k=1}^{\infty }a_{k}=\infty \).
-
There exists a continuously differentiable function \(v(\cdot):\mathbb {R}^{l}\to \mathbb {R}\) such that v(x)≠0 for ∀x≠0,v(0)=0 and for any 0<r1<r2<∞
$$\begin{array}{*{20}l} \sup_{k}\sup_{r_{1}\le ||x-\theta_{k}||\le r_{2}}f_{k}^{T}(x)v_{x}(x-\theta_{k})<-a, \end{array} $$where a is a positive constant possibly depending on r1,r2. A constant r>η exists such that
$$\begin{array}{*{20}l} \sup_{||y||\le\eta}v(y)<\sup_{||x||=r}v(x), \end{array} $$where η is an unknown constant specified later in Lemma 1.
-
The class of functions {fi,k(·)}k≥0 is equi-continuous for i=1,…,N, i.e., for any fixed i and any ε>0, there exists δ>0 such that
$$ ||f_{i,k}(x)-f_{i,k}(y)||\le\epsilon \quad\forall k,\quad \text{whenever}\quad ||x-y||\le\delta, $$where δ only depends on ε. Furthermore, for ∀c>0, there exists a constant α(c) such that ||fi,k(θk+ν)||<α(c) for ∀ν with \(||\nu ||\le c, \forall i\in \mathcal {V},k=1,2,\dots \)
-
a) The adjacent matrices W(k) ∀k≥0 are doubly stochastic;
b) There exists a constant 0<κ<1 such that
$$\begin{array}{*{20}l} w_{ij}(k)\ge\kappa\quad\forall j\in N_{i}(k)\quad \forall i\in\mathcal{V}\quad\forall k\ge 0, \end{array} $$c) The digraph \(\mathcal {G}_{\infty }=\{\mathcal {V},\mathcal {E}_{\infty }\}\) is strongly connected, where
$$\begin{array}{*{20}l} {}\mathcal{E}_{\infty}=\{(j,i):(j,i)\in\mathcal{E}(k)\text{ for infinitely many indices }k\}, \end{array} $$d) There exists a positive integer B such that
$$\begin{array}{*{20}l} (j,i)\in\mathcal{E}(k)\cup\mathcal{E}(k+1)\cup\cdots\cup\mathcal{E}(k+B-1) \end{array} $$for all \((j,i)\in \mathcal {E}_{\infty }\) and any k≥0.
-
For any \(i\in \mathcal {V}\), the noise sequence {εi,k+1}k≥0 is such that
$${\lim}_{T\to 0}\limsup_{k\to\infty}\frac{1}{T}||\sum_{m=n_{k}}^{m(n_{k},t_{k})}a_{m}\epsilon_{i,m}||=0,\quad\forall t_{k}\in[0,T]$$where \(m(k,T)\triangleq \max \left \{m:\sum _{i=k}^{m}a_{i}\le T\right \}\) and {nk} denotes the indices of any convergent subsequence \(\left \{x_{i,n_{k}}-\theta _{n_{k}}\right \}\).
-
\(g_{k}(\cdot):\mathbb {R}^{l}\to \mathbb {R}^{l}\) is equi-continuous with respect to k and is such that \(||d_{k}(x)||\le \gamma _{k}||x-\theta _{k}|| \forall x,k=1,2,\dots \), where
$$d_{k}(x)\triangleq g_{k}(x)-g_{k}(\theta_{k})-(x-\theta_{k}),$$γk=o(ak), and \(\sum _{k=1}^{\infty }\gamma _{k}<\infty \).
-
||ξk||=o(ak) and \(\sum _{k=1}^{\infty }||\xi _{k}||<\infty \).
A1 and A2 are the standard assumptions for stochastic approximation. A3 implies the local boundedness of the functions fi,k(·). Notice that the upper bound α(c) in A3 should be uniform with respect to k.
A4 describes the information exchanging among agents. We refer to [9] for the detailed explanation. Set \(\Phi (k,k+1)\triangleq \mathbf {I}_{N}\) and
By Proposition 1 in [9] it follows that there exist constants c>0 and 0<ρ<1 such that
Notice that in A5 b), the noise condition is required only along the indices of any convergent subsequence \(\left \{x_{i,n_{k}}-\theta _{n_{k}}\right \}\). As to be seen in the next section, this makes the convergence analysis much easier compared with requiring the noise condition to hold along the whole sequence.
In A6, dk(x) measures the difference between the estimation error xi,k−θk and the prediction error gk(xi,k)−gk(θk). This assumption implies that the dynamic of the root, i.e. gk(·), will tend to be a linear function as time k goes to infinity. For example, if the dynamics of the changing roots is gk(x)=x+c, then A6 holds with γk=0.
3.2 Main result
Set \(\text {col}\left \{x_{1},\dots,x_{m}\right \}\triangleq \left (x_{1}^{T},\dots,x_{m}^{T}\right)^{T}\), and define
Further, we denote the disagreement vector of Xk by \(X_{\bot,k}\triangleq D_{\bot }X_{k}\) with \(D_{\bot }\triangleq \left (\mathbf {I}_{N}-\frac {\mathbf {1}\mathbf {1}^{T}}{N}\right)\otimes \mathbf {I}_{l}\). Define \(x_{k}\triangleq \frac {1}{N}\sum _{i=1}^{N}x_{i,k}\), the average of all agents’ estimates at time k. Define \(\Delta _{i,k}\triangleq x_{i,k}-\theta _{k}, \Lambda _{k}\triangleq X_{k}-\Theta _{k}\), and \(\Delta _{k}\triangleq x_{k}-\theta _{k}\).
Theorem 1
Let {xi,k} be the estimates produced by (3)–(7) with an arbitrary initial value xi,0. Assume A1-A4 and A6 hold. If for a fixed sample ω, A5 holds for all agents, and A7 holds, then for this ω, the following assertion takes place:
i) There exists a positive integer k0 depending on ω such that
or in the compact form
for any k≥k0;
ii)
Theorem 1 i) shows that the truncation ceases after a finite number of steps. This implies that the difference between the estimate xi,k and hk−1(x∗) is bounded, which is desirable as to be shown in Lemma 1. Before we move on to the proof of Theorem 1, we first show that the truncation mechanism is reasonable.
Lemma 1
If A6 and A7 hold, the sequence {hk(x∗)−θk+1} is bounded for any x∗.
Proof
By A6 and A7 from (2) it follows that
where \(\prod _{i=1}^{\infty }(1+\gamma _{i})<\infty \) is implied by \(\sum _{j=1}^{\infty }\gamma _{j}<\infty \). □
As we mentioned in Section 2, Lemma 1 shows that the distance between {hk−1(x∗)} and {θk} is bounded. Since we hope the estimate {xi,k} generated by the algorithm (3)–(7) track the root {θk}, the truncation mechanism \(\mathbb {I}_{\left [||x^{\prime }_{i,k+1}-h_{k}(x^{*})||\le M_{\hat {\sigma }_{i,k}}\right ]}\) is intuitively reasonable.
4 Auxiliary sequences
The next two sections of this paper focus on the proof of Theorem 1. But prior to analyzing {xi,k}, we need to introduce two auxiliary sequences \(\left \{\tilde {x}_{i,k}\right \}\) and \(\left \{\tilde {\epsilon }_{i,k}\right \}\) for each agent \(i\in \mathcal {V}\). The motivation of constructing these two sequences comes from the character of distributed algorithm with expanding truncation. Recall the convergence analysis of stochastic approximation algorithm with expanding truncation (SAAWET) [28]. The key step is to show that the truncations cease after a finite number of steps, therefore, the boundedness of the estimates is established. If the number of truncations increases unboundedly, then the estimate is pulled back to x∗ infinitely many times. This produces a convergent subsequence from the estimate sequence. Then a contradiction can be shown analysing the property along this subsequence, which proves the boundedness of the estimates.
Although the problem in this paper is different from the one in [28] since the regression function is time-varying in this paper, we use the same approach to prove the boundedness of the estimates. Notice the distributed algorithm with expanding truncation (3)–(7). When \({\lim }_{k\to \infty }\sigma _{i,k}=\infty ~\forall i\in \mathcal {V}\), the estimate xi,k is pulled back to hk−1(x∗) infinitely times. By lemma 1, we know that ||hk−1(x∗)−θk|| is bounded. So {Δi,k} contains a convergent subsequence. However, {Δk} may still not contain any convergent subsequences. This is because truncation may occur at different times for different \(i\in \mathcal {V}\). Therefore, the analysis approach used for SAAWET cannot directly be applied to the algorithm (3)–(7).
To overcome this difficulty, we introduce the auxiliary sequences \(\left \{\tilde {x}_{i,k}\right \}\) and \(\left \{\tilde {\epsilon }_{i,k}\right \}\). As to be shown, the auxiliary sequences \(\left \{\tilde {x}_{i,k}\right \}\) satisfies the recursions (19)–(21), for which the number of truncation at time k for all agents is the same and the estimates \(\tilde {x}_{i,k}\) for all the agents are pulled back to hk−1(x∗) when σk>σk−1. The auxiliary noise \(\left \{\tilde {\epsilon }_{i,k}\right \}\) satisfies a condition similar to A5 b). These make the analysis for (19)–(21) feasible.
It is shown below that the important feature of the auxiliary sequences consists in that \(\left \{\tilde {x}_{i,k}\right \}\) and {xi,k} coincide in a finite number of steps, which means that the convergence of these two sequences is equivalent.
Denote by \(\tau _{i,m}\triangleq \inf \left \{k:\sigma _{i,k}=m\right \}\) the smallest time when the truncation number of agent i has reached m, by \(\tau _{m}\triangleq \min _{i\in \mathcal {V}}\tau _{i,m}\) the smallest time when at least one of agents has its truncation number reached m, and by
the largest truncation number among all agents at time k. Set \(\tilde {\tau }_{i,m}\triangleq \tau _{i,m}\land \tau _{m+1}\), where a∧b= min{a,b}.
For any \(i\in \mathcal {V}\), define the auxiliary sequences \(\left \{\tilde {x}_{i,k}\right \}_{k\ge 0}\) and \(\left \{\tilde {\epsilon }_{i,k}\right \}_{k\ge 0}\) as follows:
where m is an integer.
Note that for the considered sample ω there exists a unique integer m≥0 corresponding to an integer k≥0 such that τm≤k<τm+1. By definition \(\tilde {\tau }_{i,m}\le \tau _{m+1}~\forall i\in \mathcal {V}\). So, \(\left \{\tilde {x}_{i,k}\right \}_{k\ge 0}\) and \(\left \{\tilde {\epsilon }_{i,k}\right \}_{k\ge 0}\) are uniquely determined by the sequences {xi,k}k≥0 and {εi,k}k≥0.
Lemma 2
For k∈[τm,τm+1), the following assertions hold:
Proof
i) Since σi,k<m, by the definition of τi,m and the fact that k∈[τm,τm+1), we know τi,m>k. Thus, \(\tilde {\tau }_{i,m}=\tau _{i,m}\land \tau _{m+1}>k\). Hence, we conclude (15) from (13).
ii) Since σi,k=m, by definition we have τi,m≤k. Hence \(\tilde {\tau }_{i,m}=\tau _{i,m}\land \tau _{m+1}=\tau _{i,m}\le k\). So by (14) we conclude (16).
iii) By τm≤k<τm+1 we know σj,k≤m. We consider two cases: σj,k=m, and σj,k<m. 1) For σj,k=m, since σj,k−1<m, we know that truncation happens at time k for agent j. Truncation happens only when one of the following cases hold true: a) \(\sigma _{j,k-1}<\hat {\sigma }_{j,k-1}\). For this case, by (3) we have \(x^{\prime }_{j,k}=h_{k-1}(x^{*})\), hence by (4) we have xj,k=hk−1(x∗); b) \(\|x^{\prime }_{j,k}-h_{k-1}(x^{*})\|> M_{\hat {\sigma }_{j,k-1}}\). For this case, by (4) we have xj,k=hk−1(x∗). In conclusion, when σj,k=m holds true, we have xj,k=hk−1(x∗). Furthermore, from (16) it follows that \(\tilde {x}_{j,k}=x_{j,k}\) if σj,k=m. So, we have \(\tilde {x}_{j,k}=h_{k-1}(x^{*})\). 2) For σj,k<m, from (15) we have \(\tilde {x}_{j,k}=h_{k-1}(x^{*})\).
iv) From k∈[τm,τm+1] we know σk=m. Hence from σk+1=m+1 by definition we have τm+1=k+1, and k+1∈[τm+1,τm+2). By σk=m we see \(\sigma _{j,k}<m+1~\forall j\in \mathcal {V}\). Then we derive (18) by (17). □
Lemma 3
The auxiliary sequences \(\{\tilde {x}_{i,k}\},~\{\tilde {\epsilon }_{i,k}\}\) defined by (13)(14) satisfy the following recursions:
Proof
We prove this by induction.
First we prove (19)–(21) for k=0. Since 0∈[τ0,τ1) and \(\sigma _{0}=0~\forall i\in \mathcal {V}\), by (16) we have \(\tilde {x}_{i,0}=x_{i,0},~\tilde {\epsilon }_{i,1}=\epsilon _{i,1}~\forall i\in \mathcal {V}\). Then by \(\hat {\sigma }_{i,0}=\sigma _{i,0}=0~\forall i\in \mathcal {V}\), from (3)(19) we see
Now we prove that \(\tilde {x}_{i,1}\) and σ1 generated by (19)–(21) are consistent with the definition (12)(13)(14). We consider two cases:
i) There is no truncation at time k=1, i.e., \(\sigma _{i,1}=0~\forall i\in \mathcal {V}\). Since σi,0=0, by (5) we know that \(\phantom {\dot {i}\!}||x_{i,1}^{\prime }-h_{0}(x^{*})||\le M_{0}\). Then we have \(\phantom {\dot {i}\!}x_{i,1}=x^{\prime }_{i,1}\) by (4), and \(\tilde {x}_{i,1}=\hat {x}_{i,1}, \sigma _{1}=0\) by (20)(21). Combining these with (22) it is shown that \(\tilde {x}_{i,1}=x_{i,1}~\forall i\in \mathcal {V}\), which is consistent with (14) since \(\tilde {\tau }_{i,0}\le 1<\tau _{1}\). By (12) we see \(\sigma _{1}=\max _{i\in \mathcal {V}}\sigma _{i,1}=0\), which is consistent with the one derived from (21).
ii) There is a truncation at k=1 for agent i0, i.e., \(\sigma _{i_{0},1}=1\). Then by (4)(5) we have \(x_{i_{0},1}=h_{0}(x^{*}), ||x_{i_{0},1}^{\prime }-h_{0}(x^{*})||>M_{0}\). Hence \(||\hat {x}_{i_{0},1}-h_{0}(x^{*})||>M_{0}\) by (22). From (20)(21) we have \(\hat {x}_{i,1}=h_{0}(x^{*})~\forall i\in \mathcal {V}\) and σ1=1. By (12) from \(\sigma _{i_{0},1}=1\) we derive σ1=1. Since 0∈[τ0,τ1) and σ1=1, by (18) we have \(\tilde {x}_{i,1}=h_{0}(x^{*})~\forall i\in \mathcal {V}\). Thus, \(\tilde {x}_{i,1}\) and σ1 defined by (13)(14)(12) are consistent with those generated by (19)–(21).
In summery, we prove the lemma for k=0.
By induction, we assume (19)–(21) hold for \(k=0,1,\dots,p\). At a fixed sample ω for a given integer p there exists a unique integer m such that τm≤p<τm+1. Now we aim to show that (19)–(21) hold for k=p+1. Before this, we first express \(\hat {x}_{i,p+1}~\forall i\in \mathcal {V}\) produced by (19) for the following two cases:
Case 1: σi,p<m. Since p∈[τm,τm+1), by (15) we see
From σi,p<m it follows that σj,p−1<m ∀j∈Ni(p) by (5). Then by (17) we have \(\tilde {x}_{i,p}=h_{p-1}(x^{*})~\forall j\in N_{i}(p)\), which combining with (19)(23) shows
Case 2: σi,p=m. By τm≤p<τm+1 we have \(\sigma _{j,p}\le m~\forall j\in \mathcal {V}\) and hence by (6) we derive
Then by (3)
From σi,p=m and p∈[τm,τm+1), by (16) it can be shown that
Substituting (27) into (19), by (26) we know that
So we have expressed \(\hat {x}_{i,p+1}~\forall i\in \mathcal {V}\) produced by (19) for the two cases above.
Since τm≤p<τm+1, we have σp<m+1 and hence σp+1≤m+1. From τm≤p it follows that σp=m and σp+1≥m, hence m≤σp+1≤m+1.
Now we show that \(\tilde {x}_{i,p+1}\) and σp+1 generated by (19)–(21) are consistent with their definitions (12)(13)(14). We prove this for two cases σp+1=m+1 and σp+1=m.
Case 1: σp+1=m+1. We first show
for the following two cases: 1) σi,p<m and σj,p<m ∀j∈Ni(p). For this case by (6) we have \(\hat {\sigma }_{i,p}<m\), and hence \(\sigma _{i,p+1}\le \hat {\sigma }_{i,p}+1\le m\) by (5). 2) σi,p<m and σj,p=m for some j∈Ni(p). For this case we derive \(\hat {\sigma }_{i,p}=m, x^{\prime }_{i,p+1}=h_{p}(x^{*})\) by (3)(6). So, by (5) we derive \(\sigma _{i,p+1}=\hat {\sigma }_{i,p}=m\). Thus, σi,p+1≤m when σi,p<m. Hence (29) holds. Furthermore, this means that
Since we are considering the case where σp+1=m+1, by definition we know that there exists some agent \(i_{0}\in \mathcal {V}\) such that \(\sigma _{i_{0},p+1}=m+1\). Then \(\sigma _{i_{0},p}=m\) by (30), and hence \(\hat {\sigma }_{i_{0},p}=m\) from (25). Then from \(\sigma _{i_{0},p+1}=m+1\) by (5) we know that \(||x^{\prime }_{i_{0},p+1}-h_{p}(x^{*})||>M_{m}\). So from (20)(21) we derive \(\tilde {x}_{i,p+1}=h_{p}(x^{*})~\forall i\in \mathcal {V}\) and σp+1=m+1, which is consistent with the σp+1 defined by (12). Since σp+1=m+1 and p∈[τm,τm+1), by (18) we see that \(\tilde {x}_{i,p+1}=h_{p}(x^{*})~\forall i\in \mathcal {V}\), which is consistent with that generated by (19)–(21).
Case 2: σp+1=m. In this case \(\sigma _{i,p+1}\le m~\forall i\in \mathcal {V}\). By (25), from (4)(5) we see that
So, by (28) we derive
From (24) we have \(||\hat {x}_{i,p+1}-h_{p}(x^{*})||=0\le M_{m}~\forall i:\sigma _{i,p}<m\), which incorporating with (32) yields \(||\hat {x}_{i,p+1}-h_{p}(x^{*})||\le M_{m}\forall i\in \mathcal {V}\). Then from (20) it follows
which means that σp+1 is consistent with that defined by (12).
It remains to show that \(\tilde {x}_{i,p+1}\) generated by (19)–(21) is consistent with that defined by (13)(14). We consider two cases: 1) σi,p=m. For this case, by (25)(31)(32) we see \(\tilde {x}_{i,p+1}=x_{i,p+1}~\forall i:\sigma _{i,p}=m\). By σp+1=m we see p∈[τm,τm+1), and hence \(\tilde {x}_{i,p+1}=x_{i,p+1}\) by (16). So the consistency assertion holds for any i with σi,p=m. 2) σi,p<m. For this case, from σp+1=m we see p+1∈[τm,τm+1), and hence by σi,p<m and (17) we know \(\tilde {x}_{i,p+1}\) defined by (13)(14) is equal to hp(x∗). By (24)(33) we derive \(\tilde {x}_{i,p+1}=h_{p}(x^{*})\). So the consistency assertion holds for i with σi,p<m too.
In summery, \(\tilde {x}_{i,p+1}\) and σp+1 generated by (19)–(21) are consistent with their definitions (12)(13)(14). So the induction is complete. □
Lemma 4
Assume A4 holds. Then
i)
where di,j is the length of the shortest directed path from i to j in \(\mathcal {G}_{\infty }\), and B is the positive integer given in A4 d).
ii)
where \(D\triangleq \max _{i,j\in \mathcal {V}}d_{i,j}\).
Proof
i) Since \(\mathcal {G}_{\infty }\) is strongly connected by A4 c), for any \(j\in \mathcal {V}\) there exists a sequence of nodes \(i_{1},i_{2},\dots,i_{d_{i,j}-1}\) such that \((i,i_{1})\in \mathcal {E}_{\infty },(i_{1},i_{2})\in \mathcal {E}_{\infty },\dots,\left (i_{d_{i,j}-1,j}\right)\in \mathcal {E}_{\infty }\).
Noticing that \((i,i_{1})\in \mathcal {E}_{\infty }\), by A4 d) we have
Therefore, there exists a positive integer k′∈[k,k+B−1] such that \((i,i_{1})\in \mathcal {E}(k^{\prime })\). So, \(i\in N_{i_{1}}(k^{\prime })\), and hence by (6) and (5) we have
Repeat this procedure, we can obtain \(\sigma _{i_{2},k+2B}\ge \sigma _{i_{1},k+B}\ge \sigma _{i,k}\). Finally we can reach (34).
ii) For some m≥1, let τm=k1. Then there exists an i such that τi,m=k1. By (34) we have \(\sigma _{j,k_{1}+Bd_{i,j}}\ge \sigma _{i,k_{1}}=m~\forall j\in \mathcal {V}\).
For the case where \(\sigma _{j,k_{1}+Bd_{i,j}}=m~\forall j\in \mathcal {V}\), we have \(\tau _{j,m}\le k_{1}+Bd_{i,j}~\forall j\in \mathcal {V}\). By noticing τm=k1, by the definition of \(\tilde {\tau }_{j,m}\) we have (35):
For the case where \(\sigma _{j,k_{1}+Bd_{i,j}}>m\) for some \(j\in \mathcal {V}\), we have τm+1≤k1+Bdi,j for some \(j\in \mathcal {V}\), and hence τm+1≤τm+BD. Again, we obtain (35):
□
Corollary 1
If \(\sigma _{k}\xrightarrow [k\to \infty ]\infty \), then \({\lim }_{k\to \infty }\sigma _{i,k}=\infty ~\forall i\in \mathcal {V}\).
This corollary can be easily obtained from (34).
Lemma 5
Assume A5 holds at the sample path ω under consideration, A1, A3,A6, and A7 hold. Then for this ω
along indices {nk} whenever \(\left \{\widetilde {\Lambda }_{n_{k}}\right \}\) converges at ω, where \(\tilde {\epsilon }_{k}\triangleq col\left \{\tilde {\epsilon }_{1,k},\dots,\tilde {\epsilon }_{N,k}\right \}, \tilde {X}_{k}\triangleq col\left \{\tilde {x}_{1,k},\dots,\tilde {x}_{N,k}\right \}\), and \(\widetilde {\Lambda }_{k}\triangleq \tilde {X}_{k}-\Theta _{k}\).
Proof
It suffices to show
along indices {nk} whenever \(\left \{\tilde {x}_{i,n_{k}}-\theta _{n_{k}}\right \}\) converges at sample ω where A5 b) holds for agent i.
We consider two cases:
Case 1: \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \). From definition we can obtain
By (35) we have \(\tilde {\tau }_{i,\sigma }\le \tau _{\sigma }+BD\), hence by (14)(38)
So,
for any t>0 and any sufficiently large k. Then by A5 b) we prove (37).
Case 2: \({\lim }_{k\to \infty }\sigma _{k}=\infty \). In this case we prove (37) for three separate cases:
i): \(\tilde {\tau }_{i,\sigma _{n_{p}}}\le n_{p}\). For this case, \(\left [n_{p},\tau _{\sigma _{n_{p}}+1}\right)\subset \left [\tau _{i,\sigma _{n_{p}}},\tau _{\sigma _{n_{p}}+1}\right)\). So by (14) we have
Thus, for any tp∈[0,T]
Notice that by (40), \(\tilde {x}_{i,n_{p}}=x_{i,n_{p}}\), and the indices {np} is taken when \(\{\tilde {x}_{i,n_{p}}-\theta _{n_{p}}\}\) is a convergent subsequence. So \(\{x_{i,n_{p}}-\theta _{n_{p}}\}\) is a convergent subsequence as well. It can be seen \(\sum _{s=n_{p}}^{m(n_{p},t_{p})\land (\tau _{\sigma _{n_{p}}+1}-1)}a_{s}\le \sum _{s=n_{p}}^{m(n_{p},t_{p})}a_{s}\le t_{p}\le T\). Hence, from (41) and A5 b) we conclude (37).
ii): \(\tilde {\tau }_{i,\sigma _{n_{p}}}>n_{p}\) and \(\tilde {\tau }_{i,\sigma _{n_{k}}}=\tau _{\sigma _{n_{k}+1}}\). By definition of τk and σk we have \(\tau _{\sigma _{k}}\le k\), and hence \(\tau _{\sigma _{n_{p}}}\le n_{p}\). Then \([n_{p},\tau _{\sigma _{n_{p}+1}})\subset [\tau _{\sigma _{n_{p}}},\tilde {\tau }_{i,\sigma _{n_{p}}})\), and hence by (13) we have
From \(\tilde {\tau }_{i,\sigma _{n_{p}}}=\tau _{\sigma _{n_{p}}+1}\) by (35) we know that \(\tau _{\sigma _{n_{p}}+1}\le \tau _{\sigma _{n_{p}}}+BD\le n_{p}+BD\). Then for any tp∈[0,T], utilizing A1, A3 and Lemma 1 we have
and hence (37) holds for this case.
iii): \(\tilde {\tau }_{i,\sigma _{n_{p}}}>n_{p}\) and \(\tilde {\tau }_{i,\sigma _{n_{p}}}<\tau _{\sigma _{n_{p}}+1}\). For this case from definition we know that \(\tilde {\tau }_{i,\sigma _{n_{p}}}=\tau _{i,\sigma _{n_{p}}}\). So by (35) we have \(\tau _{i,\sigma _{n_{p}}}\le \tau _{\sigma _{n_{p}}}+BD\). Noticing \(\tau _{\sigma _{n_{p}}}\le n_{p}\), we conclude that
So, \([n_{p},\tau _{i,\sigma _{n_{p}}})\subset [\tau _{\sigma _{n_{p}}},\tilde {\tau }_{i,\sigma _{n_{p}}})\). From this and \(\tilde {\tau }_{i,\sigma _{n_{p}}}=\tau _{i,\sigma _{n_{p}}}\), by (13)(14) we derive
Consequently, for any tp∈[0,T]
Analyze the first term at the right hand of (45):
From the definition of τi,k, the truncation number of agent i at time \(\tau _{i,\sigma _{n_{p}}}\) is \(\sigma _{n_{p}}\) while it’s smaller than \(\sigma _{n_{p}}\) at time \(\tau _{i,\sigma _{n_{p}}}-1\). So by the algorithm (3)-(6) we know \(x_{i,\tau _{i,\sigma _{n_{p}}}}=h_{\tau _{i,\sigma _{n_{p}}}-1}(x^{*})\). Notice the second term at the right hand of (45). If we can show \(\{h_{\tau _{i,\sigma _{n_{p}}}-1}(x^{*})-\theta _{\tau _{i,\sigma _{n_{p}}}}\}\) is convergent, combining it with the fact \(\sum _{s=\tau _{i,\sigma _{n_{p}}}}^{m(n_{p},t_{p})\land \tau _{\sigma _{n_{p}}+1}-1}a_{s}\le \sum _{s=n_{p}}^{m(n_{p},t_{p})}a_{s}\le t_{p}\), from A5 we can conclude that the second term at the right hand of (45) tends to zero as p→∞.
We show that {hk(x∗)−θk+1} is a convergent sequence by proving that the sequence is a Cauchy sequence. For two different integer j>i>0, we see
where the last inequality comes from A6 and Lemma 1. By A6 and A7 we know that \(\sum _{k=1}^{\infty }\gamma _{k}<\infty \) and \(\sum _{k=1}^{\infty }\xi _{k}<\infty \). Thus, for ∀ε>0, we can find a sufficiently large N>0 such that ||hj(x∗)−θj+1−hi(x∗)+θi+1||<ε for ∀i,j>N, which means that {hk(x∗)−θk+1} is a Cauchy sequence. Furthermore, we can prove that (37) holds for case iii).
Since one of case i), ii), iii) must take place for the case \({\lim }_{k\to \infty }\sigma _{k}=\infty \), we can conclude that (37) holds in Case 2.
Combining Case 1 and Case 2, we conclude (36). □
Corollary 2
In (38) we show that τσ+1=∞ when \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \). So, if \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \), by (20) we know that \(\{\tilde {x}_{i,k}\}\) and \(\{x_{i,k}\}, \{\tilde {\epsilon }_{i,k}\}\) and {εi,k} coincide in a finite number of steps.
5 Proof of the main result
Define:
and
Since W(k) are doubly stochastic, by the property of Kronecker product (A⊗B)(C⊗D)=(AC)⊗(BD) we know that for ∀k≥s−1
The following lemma characterizes the closeness of the auxiliary sequence \(\{\widetilde {\Lambda }_{k}\}_{k\geq 1}\) along its convergent subsequence \(\{\widetilde {\Lambda }_{n_{k}}\}\).
Lemma 6
Assume A1, A3, A4, and A6 hold. Further, for a fixed sample ω, assume A5 and A7 hold for all the agents. Let \(\{\widetilde {\Lambda }_{n_{k}}\}\) be a convergent subsequence of \(\{\widetilde {\Lambda }_{k}\}\) for ω. \(\widetilde {\Lambda }_{n_{k}}\xrightarrow [k\to \infty ]{}\widetilde {\Lambda }\). Then for this ω there is a T>0 such that for all sufficiently large k and any Tk∈[0,T]
for any \(m=n_{k},\dots,m(n_{k},T_{k})\), and
where \(\tilde {x}_{k}\triangleq \frac {1}{N}\sum _{k=1}^{N}\tilde {x}_{i,k}, \widetilde {\Delta }_{k}\triangleq \tilde {x}_{k}-\theta _{k}\), and \(\widetilde {\Delta }_{i,k}\triangleq \tilde {x}_{i,k}-\theta _{k}\).
Proof
Consider a fixed sample path ω where A5 and A7 hold.
Let \(C>||\widetilde {\Lambda }||\). There exists an integer kC>0 such that
From Lemma 5 we know that there exist constants T1>0 and k0>kC such that
Define
where c and ρ are given by (9). Select T such that
For any k≥k0 and any Tk∈[0,T] define
So from (51) and (56) it follows that
We intend to prove sk>m(nk,Tk). Assume the converse that for sufficiently large k≥k0 and any Tk∈[0,T]
We first show that there exists a positive integer k1>k0 such that for any k≥k1
We prove (60) for two cases: \({\lim }_{k\to \infty }\sigma _{k}=\infty \) and \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \).
i) \({\lim }_{k\to \infty }\sigma _{k}=\infty \): From (58) we know that \(||\tilde {x}_{i,n_{k}}-\theta _{n_{k}}||\le M^{\prime }_{0}+C+1~\forall i\in \mathcal {V}\). First, we prove that for sufficiently large k, truncation does not happen at time nk+1. For any \(i\in \mathcal {V}\), we consider the following two cases:
a) \(\tilde {x}_{i,n_{k}}\) and \(\tilde {\epsilon }_{i,n_{k}+1}\) take value as (13): From (19) we have
Since A4 indicates that W(nk) is doubly stochastic, by A6, (51) and direct calculation we have the following inequalities
and hence by Lemma 1 we know \(||\hat {x}_{i,n_{k}+1}-h_{n_{k}}(x^{*})||\le \eta +2M^{\prime }_{0}+2C+3\).
b) \(\tilde {x}_{i,n_{k}}\) and \(\tilde {\epsilon }_{i,n_{k}+1}\) take value as (14): From (19) we have
By A5 a) we know that \(a_{n_{k}}\epsilon _{i,n_{k}+1}<1\) for sufficiently large k. Then, by A3, A4, A6 and (51), we have the following inequalities
and hence by Lemma 1 we know \(||\hat {x}_{i,n_{k}+1}-h_{n_{k}}(x^{*})||\le \eta +M_{1}\).
So we show that when \(||\widetilde {\Lambda }_{n_{k}}||\le M^{\prime }_{0}+C+1\), we have \(||\hat {x}_{i,n_{k}+1}-h_{n_{k}}(x^{*})||\le \eta +M_{1}\). Since {Mk} is a sequence of positive number increasingly diverging to infinity, there exits a positive integer k1>k0 such that \(M_{\sigma _{n_{k}}}>\eta +M_{1}\) for all k≥k1. Thus, we prove that truncation does not happen at time nk+1.
Notice (58) holds for j:nk≤j≤sk. So, similar to the proof above, we can prove that truncation does not happen for time \(n_{k}+1,\dots,s_{k}+1\). Then we conclude \(\phantom {\dot {i}\!}s_{k}<\tau _{\sigma _{n_{k}}+1}\).
ii) \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \): For this case there exists a positive integer k1>k0 such that \(\phantom {\dot {i}\!}\sigma _{n_{k}}=\sigma \) for all k≥k1, and hence \(\phantom {\dot {i}\!}\tau _{\sigma _{n_{k}}+1}=\infty \) by definition. Then \(\phantom {\dot {i}\!}m(n_{k},T_{k})<\tau _{\sigma _{n_{k}}+1}\), hence by (59) we know \(\phantom {\dot {i}\!}s_{k}<\tau _{\sigma _{n_{k}}+1}\). So (60) is proven.
By (56) we see Tk∈[0,T]⊂[0,T1], then from (52) we know that for sufficiently large k>k1 and any Tk∈[0,T]
By setting \(\phantom {\dot {i}\!}t_{k}=\sum _{m=n_{k}}^{s}a_{m}\) for some s∈[nk,sk], from (59) we see \(\phantom {\dot {i}\!}\sum _{m=n_{k}}^{s}a_{m}\le \sum _{m=n_{k}}^{s_{k}}a_{m}\le T_{k}\). Noticing m(nk,tk)=s, from (60) we derive \(\phantom {\dot {i}\!}m(n_{k},t_{k})\land (\tau _{\sigma _{n_{k}}+1}-1)=s\). So by (61) we know that
for sufficiently large k≥k1 and any Tk∈[0,T].
Now we consider the following recursive algorithm starting from nk:
where \(Z_{k}\triangleq \text {col}\{z_{i,k},\dots,z_{N,k}\}\). By (60) we know that (48) holds for \(m=n_{k},\dots,s_{k}-1\) for ∀k≥k1 ∀Tk∈[0,T]. Then we derive
Set \(z_{k}=\frac {\mathbf {1}^{T}\otimes \mathbf {I}_{l}}{N}Z_{k}, \widehat {\Delta }_{i,k}\triangleq z_{i,k}-\theta _{k}, \widehat {\Delta }_{k}\triangleq z_{k}-\theta _{k}\), and \(\widehat {\Lambda }_{k}\triangleq Z_{k}-\Theta _{k}\). By multiplying both sides of (63) with \(\frac {1}{N}(\mathbf {1}^{T}\otimes \mathbf {I}_{l})\), from 1TW(m)=1T and (A⊗B)(C⊗D)=AB⊗CD we derive
and hence
So
where the second inequality comes from A6, the third inequality comes from (58) A3 and A5, the fourth inequality comes from (51) (59), and the last inequality comes from (59).
Denote by Z⊥,s=D⊥Zs the disagreement vector of Zs. By multiplying both sides of (63) with D⊥ we have
Notice that D⊥(W(m)⊗Il)=D⊥(W(m)⊗Il)D⊥, so we have
By definition we know that D⊥Gm(Θm)=D⊥Θm=0, hence we have
So inductively
From (9), (51), (58), (64), A3, and A6 we can derive
Set \(\Gamma _{n}\triangleq \sum _{m=1}^{n}a_{m}\tilde {\epsilon }_{m+1}\), by (62) we know that \(\phantom {\dot {i}\!}||\Gamma _{s}-\Gamma _{n_{k}-1}||\le T_{k}\quad \forall s:n_{k}\le s\le s_{k}\). Notice
So, summing by parts with (9) we have
which incorporating with (62) produces
for sufficiently large k≥k1 and any Tk∈[0,T].
Notice \(\sum _{m=n_{k}}^{s}a_{m}\rho ^{s-m}\le \frac {1}{1-\rho }\sup _{m\ge n_{k}}a_{m}\) from \(a_{m}\xrightarrow [m\to \infty ]{}0\). Combine this with (67)(68) we have
for sufficiently large k≥k1 and any Tk∈[0,T]. Notice that \(\widehat {\Lambda }_{s}=Z_{\perp,s}+(\mathbf {1}\otimes \mathbf {I}_{l})\widehat {\Delta }_{s}\). We derive
Since \(||Z_{\perp,n_{k}}||\le 2||\widehat {\Lambda }_{n_{k}}||\le 2C\), from (65)(67) it follows that for sufficiently large k≥k1 and any Tk∈[0,T]
Therefore, from (56)(51) we know that for sufficiently large k≥k1 and any Tk∈[0,T]
Now we look back at the recursive algorithm (19). We rewrite it in the compact form as follows
where \(\widehat {X}_{k}\triangleq \text {col}\{\hat {x}_{1,k},\dots,\hat {x}_{N,k}\}\). Then by (63)(64), \(\widehat {X}_{s_{k}+1}=Z_{s_{k}+1}\). So by (71) it follows that
We now show
for sufficiently large k≥k1 and any Tk∈[0,T]. We consider the following two cases: \({\lim }_{k\to \infty }\sigma _{k}=\infty \) and \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \).
i) \({\lim }_{k\to \infty }\sigma _{k}=\infty \): Notice \(M_{\sigma _{k}}>\eta +M_{0}^{\prime }+1+C\) when k≥k1. By (20)(21) we know that \(\phantom {\dot {i}\!}\tilde {X}_{s_{k}+1}=\widehat {X}_{s_{k}+1}, \sigma _{s_{k}+1}=\sigma _{s_{k}}\). So \(\phantom {\dot {i}\!}s_{k}+1<\tau _{\sigma _{n_{k}}+1}\) by (60).
ii) \({\lim }_{k\to \infty }\sigma _{k}=\sigma <\infty \): For this case \(\phantom {\dot {i}\!}\tau _{\sigma _{n_{k}+1}}=\infty \) for all k≥k1. By (60) we see \(\phantom {\dot {i}\!}s_{k}+1<\tau _{\sigma _{n_{k}+1}}\). Then by \(\phantom {\dot {i}\!}\sigma _{n_{k}}=\sigma \) we conclude \(\phantom {\dot {i}\!}\sigma _{s_{k}+1}=\sigma _{x_{k}}=\sigma \), and hence by (20) we derive \(\phantom {\dot {i}\!}\tilde {X}_{s_{k}+1}=\widehat {X}_{s_{k}+1}\). Thus (73) holds.
From (73) we know that (48) holds for m=sk for sufficiently large k≥k1 and any Tk∈[0,T]. From \(\phantom {\dot {i}\!}\widehat {X}_{s_{k}+1}=Z_{s_{k}+1}\) by (73) we see \(\phantom {\dot {i}\!}\tilde {X}_{s_{k}+1}=Z_{s_{k}+1}\). It follows that for sufficiently large k≥k1 and any Tk∈[0,T]
which contradicts with the definition of sk. Thus (59) does not hold. So sk>m(nk,Tk) and hence (49) holds.
Since sk>m(nk,Tk), we know \(\{\widetilde {\Lambda }_{s}:n_{k}\le s\le m(n_{k},T_{k})+1\}\) is bounded. Similar to proving (60) we can be shown that \(\phantom {\dot {i}\!}m(n_{k},T_{k})<\tau _{\sigma _{k}+1}\). So (48) holds for \(\phantom {\dot {i}\!}m=n_{k},\dots,m(n_{k},T_{k})\). Similar to (65) we can prove
for sufficiently large k and any Tk∈[0,T]. Hence, (50) holds.
In conclusion, the proof of Lemma 6 is complete. □
By multiplying both sides of (48) with \(\frac {1}{N}(\mathbf {1}^{T}\otimes \mathbf {I}_{l})\), we have
Setting
We can rewrite (74) as
The following lemma gives the noise property of the sequence {ζk+1}.
Lemma 7
Assume all the conditions in Lemma 6 hold. \(\{\widetilde {\Lambda }_{n_{k}}\}\) is a convergent subsequence with limit \(\widetilde {\Lambda }\) at the considered sample ω. Then for this ω
Proof
In the proof of Lemma 6 it has been pointed out that there exists a T∈(0,1) such that \(\phantom {\dot {i}\!}m(n_{k},T)<\tau _{\sigma _{n_{k}}+1}\) for sufficiently large k. So \(\phantom {\dot {i}\!}||\sum _{s=n_{k}}^{m(n_{k},T_{k})\land (\tau _{\sigma _{n_{k}}+1}-1)}a_{s}\tilde {\epsilon }_{s+1}||=||\sum _{s=n_{k}}^{m(n_{k},T_{k})}a_{s}\tilde {\epsilon }_{s+1}||\). Thus, by Lemma 5 we can immediately derive that
Now we need to show that \(\zeta ^{(i)}_{k+1}\) also satisfies the property above for i=1,2. First we consider \(\zeta ^{(1)}_{k+1}\). We see that
where the last inequality comes from A6.
Since \({\lim }_{k\to \infty }\widetilde {\Lambda }_{n_{k}}=\widetilde {\Lambda }\), by setting \(\widetilde {\Delta }\triangleq \frac {\mathbf {1}^{T}\otimes \mathbf {I}_{l}}{N}\widetilde {\Lambda }\) we see that \({\lim }_{k\to \infty }\widetilde {\Delta }_{n_{k}}=\widetilde {\Delta }\). So by (49)(50) we conclude that \(\{||\widetilde {\Delta }_{i,s}||,s:n_{k}\le s\le m(n_{k},T_{k})\}\) and \(\{||\widetilde {\Delta }_{s}||,s:n_{k}\le s\le m(n_{k},T_{k})\}\) are bounded. Without the loss of generality we denote the bound of these two sequence by A. Then from (77) with A6 it follows
So we conclude
Finally, we consider the case i=2. Notice
Similar to the proof in Lemma 6 we know that there exist constants c3,c4,c5>0 such that for sufficiently large k
hold for ∀s:nk≤s≤m(nk,T). From A1,A6, and A7 we can assume that for sufficiently large k
Since 0<ρ<1, there exists a positive integer m′ such that \(\phantom {\dot {i}\!}\rho ^{m^{\prime }}<T\). Then \(\phantom {\dot {i}\!}\sum _{m=n_{k}}^{n_{k}+m^{\prime }}a_{m}\xrightarrow [k\to \infty ]{}0\). Thus, we have nk+m′<m(nk,T) for sufficiently large k. So from (78) it follows
where o(1)→0 as k→∞. Hence for nk+m′<m(nk,T) and sufficiently large k
where δ(T)→0 as T→0.
So
And hence \(\limsup _{k\to \infty }\frac {1}{T}||\sum _{s=n_{k}}^{m(n_{k},T_{k})}a_{s}\zeta ^{(2)}_{s+1}||=\delta (T)\), which implies that
So we complete the proof. □
Lemma 8
Assume A1-A4, A6 hold, and that A5, A7 hold at the sample path ω under consideration. Then any nonempty interval [δ1,δ2] with 0∉[δ1,δ2] cannot be crossed by \(\{v(\widetilde {\Delta }_{m_{k}}),\dots,v(\widetilde {\Delta }_{l_{k}})\}\) infinitely many times with \(\{\widetilde {\Lambda }_{m_{k}}\}\) bounded. By " [δ1,δ2] being crossed by \(\{v(\widetilde {\Delta }_{m_{k}}),\dots,v(\widetilde {\Delta }_{l_{k}})\}\)" it means that \(v(\widetilde {\Delta }_{m_{k}})\le \delta _{1}, v(\widetilde {\Delta }_{l_{k}})\ge \delta _{2}\) and \(\delta _{1}<v(\widetilde {\Delta }_{s})<\delta _{2}\quad \forall s:m_{k}<s<l_{k}\).
Proof
Assume the converse: for some interval [δ1,δ2] with 0∉[δ1,δ2], there are infinitely many crossing \(\{v(\widetilde {\Delta }_{m_{k}}),\dots,v(\widetilde {\Delta }_{l_{k}})\}\) with \(\{\widetilde {\Lambda }_{m_{k}}\}\) bounded.
By the boundedness of \(\{\widetilde {\Lambda }_{m_{k}}\}\) we can extract a convergent subsequence still denoted by \(\{\widetilde {\Lambda }_{m_{k}}\}\) with limit \({\lim }_{k\to \infty }\widetilde {\Lambda }_{m_{k}}=\widetilde {\Lambda }\). So, \({\lim }_{k\to \infty }\widetilde {\Delta }_{m_{k}}=\widetilde {\Delta }\) with \(\widetilde {\Delta }\triangleq \frac {\mathbf {1}^{T}\otimes \mathbf {I}_{l}}{N}\widetilde {\Lambda }\). By (50), letting T→0 we have \(||\widetilde {\Delta }_{m_{k}+1}-\widetilde {\Delta }_{m_{k}}||\le c_{2}T\to 0\). By the definition of crossing, \(v(\widetilde {\Delta }_{m_{k}})\le \delta _{1}<v(\widetilde {\Delta }_{m_{k}+1})\), we can obtain
So by the assumption v(x)=0⇔x=0 we know there exists a constant β such that \(||\widetilde {\Delta }||>\beta \). And hence by (50) we conclude
for sufficiently small T>0 and large k.
Setting \(\widetilde {\Delta }_{k}\) to be a vector in-between \(\widetilde {\Delta }_{m_{k}}\) and \(\widetilde {\Delta }_{m(m_{k},T)}\). From (50) it follows that \(||\widetilde {\Delta }_{k}||\le c_{2}T+||\widetilde {\Delta }||+1\) for sufficiently large k. We consider the following Taylor’s expansion
Similar to (51), we take sufficiently large k, then by A3, A6 and Lemma 7, there exists a constant c′ such that
for sufficiently large k and sufficiently small T, where j:mk≤j≤m(mk,T)−1. It follows that
Identifying r1 and r2 in A2 to \(\frac {\beta }{4}\) and \(\frac {\beta }{4}+||\widetilde {\Delta }_{j+1}||\le \frac {\beta }{4}+c_{2}T+||\widetilde {\Delta }||\), respectively, we can find a>0 such that for ∀j:mk≤j≤m(mk,T)
Noticing that for \(\forall j:m_{k}\le j\le m(m_{k},T), ||d_{j}(\tilde {x}_{j})||\le \gamma _{j}||\widetilde {\Delta }_{j}||\le \gamma _{j}(c_{2}T+||\widetilde {\Delta }||+1)\), by A6 and A7 we have
By Lemma 7 it follows
Notice that for ∀j:mk≤j≤m(mk,T)
So by the continuity of v(·) we know
From A3, A6 and (51), we’ve already utilized this inequality before
So, from (88)(89)(92) we can conclude that the first and second term of (83) is o(T) as k→∞, T→0. Combining this with (87) it follows that for sufficiently large k and sufficiently small T by (83) we have
Let k→∞ we have
Notice that by Lemma 6 we have
which implies that m(mk,T)<lk for sufficiently small T. Therefore, \(v(\widetilde {\Delta }_{m(m_{k},T)})\in [\delta _{1},\delta _{2}]\) which contradicts with (94). So, the converse assumption is not true. The proof is completed. □
Lemma 9
Assume all the assumptions required by Lemma 8 hold. Then there exists a positive integer σ such that
Proof
Assume the converse:
Then there exists a sequence of integer {nk}k≥0 such that \(\sigma _{n_{k}}=k\) and \(\sigma _{n_{k}-1}=k-1\). By the algorithm (3)–(7) we know that \(\tilde {x}_{i,n_{k}}=h_{n_{k}-1}(x^{*})~\forall i\in \mathcal {V}\). Therefore, from Lemma 1 we know that \(\{\widetilde {\Lambda }_{n_{k}}\}\) is a bounded sequence, and hence, it contains a convergent subsequence. For the sake of convenience, We denote the convergent subsequence still by \(\{\widetilde {\Lambda }_{n_{k}}\}\) with limit \(\widetilde {\Lambda }\).
Since {Mk}k≥0 is a sequence of positive numbers increasingly diverging to infinity, there exists a positive integer k0 such that
where r is given in A2 and
Now we show that under the converse assumption, \(\{\tilde {\Delta }_{n_{k}}\}\) starting from nk will exit the ball r infinitely many times. Define
Notice that \(\|\widetilde {\Lambda }_{n_{k}}\|=\sqrt {N}\eta \) by Lemma 1 and r>η from A2, we derive \(\|\widetilde {\Lambda }_{n_{k}}\|<\sqrt {N}r\). Hence from (99) (100) we have nk<lk<mk. By the definition of lk we know that \(\{\widetilde {\Lambda }_{l_{k}}\}\) is bounded, then there exists a convergent subsequence denoted still by \(\{\widetilde {\Lambda }_{l_{k}}\}\).
By Lemma 6 there exist constants \(M_{0}^{\prime }>0\) defined by (53) with \(C=2\sqrt {N}r+2, c_{1}>0\) defined by (54), c2>0 defined by (55), and 0<T<1 with c1T<1 such that
for sufficiently large k≥k0. Then for sufficiently large k≥k0 we have
Then m(lk,T)<nk+1 for sufficiently large k≥k0 by (97) and the definition of nk.
From (101) by the definition of mk (99), we conclude m(lk,T)+1<mk for sufficiently large k≥k0. Then by (99) (100) we know that for sufficiently large k≥k0
holds for m:lk≤m≤m(lk,T).
Since 0<ρ<1, there exists a positive integer m0 such that \(\phantom {\dot {i}\!}4c\rho ^{m_{0}}<1\). Then \(\sum _{m=l_{k}}^{l_{k}+m_{0}}a_{m}\xrightarrow [k\to \infty ]{}0\) by A1, and hence lk+m0<m(lk,T)<nk+1 for sufficiently large k≥k0. So, from (102) it can be seen that for sufficiently large k≥k0 we have
Notice that \(\{\widetilde {\Lambda }_{m+1}:l_{k}\le m\le m(l_{k},T)\}\) is bounded, similarly to (67) (69) we know that for sufficiently large k≥k0
Since \(\phantom {\dot {i}\!}4c\rho ^{m_{0}}<1, c_{1}T<1\) and \(a_{k}\xrightarrow [k\to \infty ]{}0\), it follows that
for sufficiently large k≥k0. By noticing \((\mathbf {1}\otimes \mathbf {I}_{l})\tilde {\Delta }_{l_{k}+m_{0}}=\widetilde {\Lambda }_{l_{k}+m_{o}}-\tilde {X}_{\bot,l_{k}+m_{0}}\), from (103) (104) we conclude that
Therefore, \(\|\tilde {\Delta }_{l_{k}+m_{0}}\|>r\). So we prove that \(\{\tilde {\Delta }_{n_{k}}\}\) starting from nk will exit the ball r infinitely many times.
Since \(\{\widetilde {\Lambda }_{n_{k}}\}\) is convergent, we know that \(\{\widetilde {\Delta }_{n_{k}}\}\) is convergent and from Lemma 1 it follows that \(||\widetilde {\Delta }_{n_{k}}||\le \eta \). And we know that \(\{\tilde {\Delta }_{n_{k}}\}\) starting from nk will exit the ball r infinitely many times. Therefore, there exists an interval [δ1,δ2]∈(sup||y||≤ηv(y), inf||x||=rv(x)) with 0∉[δ1,δ2], where for any k, there is a sequence \(\tilde {\Delta }_{s_{k}},\dots,\tilde {\Delta }_{t_{k}}\) such that \(n_{k}\le s_{k}, v(\tilde {\Delta }_{s_{k}})\le \delta _{1}, \delta _{1}<v(\tilde {\Delta }_{j})<\delta _{2}\) for ∀j:sk<j<tk and \(v(\tilde {\Delta }_{t_{k}})>\delta _{2}\). In other words, the values of v(·) at sequence \(\{\tilde {\Delta }_{s_{k}},\dots,\tilde {\Delta }_{t_{k}}\}\) cross the interval [δ1,δ2] infinitely many times with \(\|\widetilde {\Lambda }_{m_{k}}\|<r\). This contradicts with Lemma 8. So the proof is done. \({\lim }_{k\to \infty }\sigma _{k}<\infty \) is indeed true. □
Lemma 10
Assume all the assumptions required by Lemma 9 hold. Then
Proof
From Lemma 9 it follows that
By Lemma 4 we know \(\tilde {\tau }_{i,\sigma }=\tau _{i,\sigma }\le BD+\tau _{\sigma }\). So, by definition we know that σi,k≥σ ∀k≥BD+τσ.
In conclusion, we have \(\sigma _{i,k}=\sigma \quad \forall k\ge BD+\tau _{\sigma }\quad \forall i\in \mathcal {V}\). The proof is completed. □
By the definition of the auxiliary sequence \(\{\tilde {x}_{i,k}\}\), we can see that Lemma 10 indicates the fact that \(\{\tilde {x}_{i,k}\}\) and {xi,k} coincide in a finite number of steps.
Proof of Theorem 1: By (95) and (106), there exists a positive integer σ depending on ω such that
and hence by (3)
by (5) \(||x^{\prime }_{i,k+1}-h_{k}(x^{*})||\le M_{\sigma }\) and by (4) \(x_{i,k+1}=x^{\prime }_{i,k+1}\) for any k≥k0 and any \(i\in \mathcal {V}\). So, we have proved the assertion i).
Multiply (11) by D⊥ from left, we derive
where the third equality comes from the fact that (W(k)⊗Il)Gk(Xk)=(W(k)⊗Il)D⊥ and D⊥Gk(Xk)=D⊥Xk=0. So, for any k≥k0 by induction we have
From Lemma 10 we know the number of truncations is finite. So, {xi,k−hk−1(x∗)}k≥1 is bounded. Furthermore, Lemma 1 shows that {hk−1(x∗)−θk}k≥1 is bounded as well. Therefore, {xi,k−θk}k≥1 is bounded. By assumption A6, A7 we can take sufficiently large k1>k0 such that ||ξk||≤1,γk≤1 ∀k≥k1. So, for sufficiently large k, there exists constants \(c_{6},c_{7},c_{8},c^{\prime }_{8},c_{9}>0\) such that
Notice that for any ε>0, there exists integer k2>k1 such that γk<ε. We can derive
Therefore, the second and third term at the right-hand side of (109) tends to zero as k→∞. Similarly, the last term of (109) also tends to zero since \({\lim }_{k\to \infty }a_{k}\epsilon _{k+1}=0\). The first term of (109) tends to zero as k→∞ as well since 0<ρ<1. So, we conclude that
We now show the convergence of \(\{v(\widetilde {\Delta }_{k})\}\). Since
we aim to prove v1=v2. Assume the converse: v1<v2. Then there exists an interval [δ1,δ2]∈(v1,v2) such that 0∉[δ1,δ2]. \(v(\widetilde {\Delta }_{k})\) crosses the interval [δ1,δ2] infinite many times. By Lemma 9 and (13)–(14) we know \(\widetilde {\Delta }_{k}\) is bounded, so, \(\widetilde {\Lambda }_{k}\) is bounded as well. This contradicts with Lemma 8. Therefore, \(\{v(\widetilde {\Delta }_{k})\}\) is convergent.
Finally, we show \(\widetilde {\Delta }_{k}\xrightarrow [k\to \infty ]{}0\). Assume the converse. Then there exists a convergent subsequence \(\{\widetilde {\Delta }_{n_{k}}\}\) with limit \(\widetilde {\Delta }\neq 0\). Take β>0 such that \(||\widetilde {\Delta }||>\beta \). From Lemma 6 we know for sufficiently large k and sufficiently small T we have
Similar to the proof of Lemma 8, by Taylor’s expansion there exists a>0 such that
Since \(\{v(\widetilde {\Delta }_{k})\}\) is convergent and by definition \(m(n_{k},T)\xrightarrow [k\to \infty ]{}\infty \), Let k→∞ for both sides of (110), we derive \(0<-\frac {a}{2}T\) which is impossible. So \(\widetilde {\Delta }_{k}\xrightarrow [k\to \infty ]{}0\). By Lemma 10 we know that after a finite number of steps we have \(\widetilde {\Delta }_{k}=\Delta _{k}\), and hence, \(\Delta _{k}\xrightarrow [k\to \infty ]{}0\). Combining \(\Delta _{k}\xrightarrow [k\to \infty ]{}0\) with \(X_{\bot,k}\xrightarrow [k\to \infty ]{}0\), we can conclude \(\Lambda _{k}\xrightarrow [k\to \infty ]{}0\). □
6 Numerical simulation
In this section, we apply the distributed algorithm to a distributed tracking problem and demonstrate the performance of the algorithm. Consider a maneuvering target in the 2-D plane. The state of the target θk at each time consists of four components \(\theta _{k}=[\theta _{k}^{1},\theta _{k}^{2},\theta _{k}^{3},\theta _{k}^{4}]^{T}\). They are horizontal position, horizontal velocity, vertical position, and vertical velocity, respectively. The dynamic model of the target is chosen to be a nearly constant velocity model [29], which means that the dynamic of the target is governed by:
where \(\xi _{k+1}\in \mathbb {R}^{4}\) is noise, and A is defined as
with T being the sampling interval. It can be seen that when ξk+1=0 the target follows a constant velocity movement. The goal of the network is to track this target by estimating the state θk.
Consider a sensor network \(\mathcal {G}=(\mathcal {V},\mathcal {E})\) with \(\mathcal {V}=\{1,\cdots,N\},~N=20,\) and \(\mathcal {E}=G(N,p_{N})\) being the Poisson random graphFootnote 1 with designing parameter 0≤pN≤1. We choose pN=0.25. Denote by Ni the neighbor set of agent i and by ni the cardinality of Ni. Set \(W(k)=[w_{ij}]_{i,j=1}^{N}~\forall k\ge 1\) with \(w_{ij}=\frac {1}{n_{i}}\) if agent j is in the set Ni. All agents aim to track the target state θk cooperatively. We assume for each agent, only one component of the target state can be observed with noise. To explain it in a mathematical model, the local function for agent i is defined as:
where ek is a 4-dimensional square diagonal matrix with only the kth diagonal element being 1 and every other elements being 0, i.e.,
The selection of ki will be explained later. Since the state θk is unknown to the agents, each agent can only get a noise-corrupted observation of this local function instead of the exact value. The global function can be written as:
It can be seen that while each agent can only estimate one component of θk with its own local function, θk is the unique root of the global function fk(x).
For our experiment, we take \(\xi _{k}\triangleq \frac {1}{k^{2}}v_{k}\), where {vk} is a sequence of i.i.d. random variables uniformly distributed over [−1,1], and the step-sizes \(a_{k}\triangleq \frac {20}{k}\). We let the sampling interval be \(T\triangleq 0.1\) s, the truncation bound \(M_{k}\triangleq k+80\), and \(x^{*}\triangleq [1,1,1,1]^{T}\). The initial value xi,0 for all agents is chosen from the uniform distribution over [−2,2]. Let the observation noise εi,k be the white Gaussian noise. As for the selection of ki, for agent i, if i mod 4≠0, then \(k_{i}\triangleq i\mod l\), if i mod 4=0, then \(k_{i}\triangleq 4\).
Denote by \(\{x_{i,k}\}_{k\ge 1},i\in \mathcal {V}\) the estimates given by (3)–(7) and by \(x_{k}=\frac {1}{N}\sum _{i=1}^{N}x_{i,k}\) the average of \(x_{i,k},~i\in \mathcal {V}\). In Fig. 1, the dashed lines denote the state of the moving target and the solid lines the average estimates for entries \(\{\theta _{k}^{j},~j=1,\cdots,4\}_{k\geq 1}\) of {θk}k≥1. From the figure we can see that the estimate can track the moving target successfully.
Average estimation sequences of \(\theta _{k}^{j},j=1,\cdots,l\)
7 Conclusion
The distributed root-tracking problem for a sum of time-varying regression functions over a network is considered in this paper. It is assumed that a noise-corrupted dynamic information of the roots is known to all agents in the network. Each agent updates its estimate by using the local observation, the dynamic information of the global root, and information received from its neighbors. A distributed stochastic approximation algorithm is proposed and the consensus and convergence of the estimates are established.
For future research, it is of interest to relax the conditions on the dynamic information of the global roots, and to consider the convergence results of the algorithm over an unbalanced network.
8 Notations
Availability of data and materials
Not applicable.
Notes
For the details of Poisson random graph, we refer to [30].
References
A. Jadbabaie, J. Lin, A. S. Morse, Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Trans. Autom. Control. 48(6), 988–1001 (2003).
Wei Ren, R. W. Beard, Consensus seeking in multiagent systems under dynamically changing interaction topologies. IEEE Trans. Autom. Control. 50(5), 655–661 (2005).
R. Olfati-Saber, A. J. Fax, R. M. Murray, Consensus and cooperation in networked multi-agent systems. Proc. IEEE. 95(1), 215–233 (2007).
W. Feng, H. -F. Chen, Output consensus of networked hammerstein and wiener systems. SIAM J. Control Optim.57(2), 1230–1254 (2019).
P. Yi, Y. Hong, F. Liu, Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems. Automatica. 74:, 259–269 (2016).
A. Nedić, A. Olshevsky, W. Shi, in 2018 IEEE Conf. Decis. Control (CDC). Improved convergence rates for distributed resource allocation (IEEE, 2018), pp. 172–177. https://doi.org/10.1109/cdc.2018.8619322.
Y. Kuriki, T. Namerikawa, in 2014 Amer. Control Conf.Consensus-based cooperative formation control with collision avoidance for a multi-uav system (IEEE, 2014), pp. 2077–2082. https://doi.org/10.1109/acc.2014.6858777.
S. Aeron, V. Saligrama, D. A. Castanon, Efficient sensor management policies for distributed target tracking in multihop sensor networks. IEEE Trans. Sig. Process.56(6), 2562–2574 (2008).
A. Nedic, A. Ozdaglar, Distributed subgradient methods for multi-agent optimization. IEEE Trans. Autom. Control. 54(1), 48 (2009).
A. Nedic, A. Ozdaglar, P. A. Parrilo, Constrained consensus and optimization in multi-agent networks. IEEE Trans. Autom. Control. 55(4), 922–938 (2010).
A. Simonetto, G. Leus, Distributed asynchronous time-varying constrained optimization (IEEE, 2014). https://doi.org/10.1109/acssc.2014.7094854.
F. Y. Jakubiec, A. R. D-map, Distributed maximum a posteriori probability estimation of dynamic systems. IEEE Trans. Sig. Process.61(2), 450–466 (2012).
X. Yi, X. Li, L. Xie, K. H. Johansson, Distributed online convex optimization with time-varying coupled inequality constraints. IEEE Trans. Sig. Process.68:, 731–746 (2020).
S. Shahrampour, A. Jadbabaie, Distributed online optimization in dynamic environments using mirror descent. IEEE Trans. Autom. Control. 63(3), 714–725 (2017).
D. Bajovic, J. Xavier, B. Sinopoli, J. M. F. Moura, et al., Distributed detection via gaussian running consensus: Large deviations asymptotic analysis. IEEE Trans. Sig. Process.59(9), 4381–4396 (2011).
M. Ye, H. Guoqiang, in 2015 54th IEEE Conference on Decision and Control (CDC). Distributed optimization for systems with time-varying quadratic objective functions (IEEE, 2015), pp. 3285–3290. https://doi.org/10.1109/cdc.2015.7402713.
A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, A. Ribeiro, Decentralized prediction-correction methods for networked time-varying convex optimization. IEEE Trans. Autom. Control. 62(11), 5724–5738 (2017).
M. Fazlyab, S. Paternain, V. M. Preciado, A. Ribeiro, Prediction-correction interior-point method for time-varying convex optimization. IEEE Trans. Autom. Control. 63(7), 1973–1986 (2017).
H. -F. Chen, T. E. Duncan, B. Pasik-Duncan, A kiefer-wolfowitz algorithm with randomized differences. IEEE Trans. Autom. Control. 44(3), 442–453 (1999).
H. J. Kushner, G. Yin, Asymptotic properties of distributed and communicating stochastic approximation algorithms. SIAM J. Control Optim.25(5), 1266–1290 (1987).
P. Bianchi, G. Fort, W. Hachem, Performance of a distributed stochastic approximation algorithm. IEEE Trans. Inform. Theory. 59(11), 7405–7418 (2013).
J. Lei, H. -F. Chen, Distributed stochastic approximation algorithm with expanding truncations. IEEE Trans. Autom. Control. 65(2), 664–679 (2020).
V. Dupač, A dynamic stochastic approximation method. Ann. Math. Stat., 1695–1702 (1965).
H. -F. Chen, K. Uosaki, Convergence analysis of dynamic stochastic approximation. Syst. Control Lett.35(5), 309–315 (1998).
D. Acemoglu, A. Nedic, A. Ozdaglar, in 2008 47th IEEE Conference on Decision and Control. Convergence of rule-of-thumb learning rules in social networks (IEEE, 2008), pp. 1714–1720. https://doi.org/10.1109/cdc.2008.4739167.
S. Shahrampour, S. Rakhlin, A. Jadbabaie, in Advances in Neural Information Processing Systems. Online learning of dynamic parameters in social networks, (2013). https://doi.org/10.4018/978-1-4666-1815-2.ch006.
F. Kewei, H. -F. Chen, W. Zhao, in 2018 37th Chinese Control Conference (CCC). Distributed stochastic approximation algorithm for time-varying regression function over network (IEEE, 2018), pp. 1925–1930. https://doi.org/10.23919/chicc.2018.8483554.
H. -F. Chen, Stochastic Approximation and Its Applications, vol. 64 (Springer Science & Business Media, 2002). https://doi.org/10.1007/b101987.
X. Yuan, C. Han, Z. Duan, M. Lei, in 2005 7th International Conference on Information Fusion, 2. Comparison and choice of models in tracking target with coordinated turn motion (IEEE, 2005), p. 6. https://doi.org/10.1109/icif.2005.1592032.
M. O. Jackson, Social and Economic Networks (Princeton university press, US, 2010).
Code availability
Not applicable.
Funding
This work was supported by the National Key Research and Development Program of China under Grant 2018YFA0703800 and the National Natural Science Foundation of China under Grant 61822312. This work was also supported (in part) by the Strategic Priority Research Program of Chinese Academy of Sciences under Grant No. XDA27000000.
Author information
Authors and Affiliations
Contributions
Authors’ contributions
All authors contributed to the study conception and design. Mathematical analysis was performed by Kewei Fu, Han-Fu Chen and Wenxiao Zhao. The first draft of the manuscript was written by Kewei Fu and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Authors’ information
Kewei Fu received the B.S. degree from Shandong University, China in 2015. He got his Ph.D. degree from the Institute of Systems Science (ISS), Academy of Mathematics and Systems Science (AMSS), Chinese Academy of Sciences (CAS), China in 2020. His research interests include the stochastic approximation algorithm and the distributed optimization and estimation over networks.
Han-Fu Chen graduated from the Leningrad (St. Petersburg) State University, Russia. He is a Professor of the Key Laboratory of Systems and Control of the Chinese Academy of Sciences (CAS). His research interests are mainly in stochastic systems, including system identification, adaptive control, and stochastic approximation and its applications to systems, control, and signal processing. He authored and coauthored more than 210 journal papers and eight books. He served as an IFAC Council Member (2002-2005), President of the Chinese Association of Automation (1993-2002), and a Permanent member of the Council of the Chinese Mathematics Society (1991-1999). He is an IEEE Fellow, IFAC Fellow, a Member of TWAS, and a Member of CAS.
Wenxiao Zhao received the B.Sc. degree from Shandong University, China in 2003 and the Ph.D. degree in operation research and cybernetics from the Institute of Systems Science (ISS), Academy of Mathematics and Systems Science (AMSS), Chinese Academy of Sciences (CAS), China, in 2008. He is currently a professor with AMSS, CAS. His research interests are mainly in machine learning, system identification and adaptive control, and stochastic optimization.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fu, K., Chen, HF. & Zhao, W. Distributed dynamic stochastic approximation algorithm over time-varying networks. Auton. Intell. Syst. 1, 5 (2021). https://doi.org/10.1007/s43684-021-00003-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s43684-021-00003-1