Skip to main content

Communication Reducing Diffusion LMS Robust to Impulsive Noise Using Smart Selection of Communication Nodes

Abstract

The paper proposes a smart method to remove network nodes that are highly prone to impulse noise and replacing them with a linear combination of existing more reliable nodes. This is done to reduce communication cost. To reduce the communication cost in a network, a common way is to remove some nodes randomly and then replace the intermediate estimation of these nodes by the corresponding node estimation. In this direction, the contribution of this paper is twofold. First, we suggest to remove the nodes smartly by omitting the unreliable nodes prone to impulsive noise. This is done by computing the disturbance induced in the adaptation step of the diffusion least mean square. Second, we replace the estimation of removing nodes by a linear combination of existing estimations of reliable nodes instead of just replacing by the estimation of the corresponding node. Also, the coefficients of linear combination are optimized based on the minimum disturbance principle. Furthermore, the minimum achievable disturbance is calculated mathematically and a necessary and sufficient condition for stability of the proposed algorithm is presented. Finally, the simulation results show the efficiency of the proposed method in comparison with some state-of-the-art algorithms.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

References

  1. 1.

    S. Al-Sayed, A.M. Zoubir, A.H. Sayed, Robust distributed estimation by networked agents. IEEE Trans. Signal Proc. 65(15), 3909–3921 (2017)

    MathSciNet  Article  Google Scholar 

  2. 2.

    R. Arablouei, S. Werner, Y. Huang, K. Dogancay, Distributed least mean-square estimation with partial diffusion. IEEE Trans. Signal Process. 62(2), 472–484 (2014)

    MathSciNet  Article  Google Scholar 

  3. 3.

    R. Arablouei, K. Dogancay, S. Werner, Y. Huang, Adaptive distributed estimation based on recursive least-squares and partial diffusion. IEEE Trans. Signal Process. 62(14), 3510–3522 (2014)

    MathSciNet  Article  Google Scholar 

  4. 4.

    R. Arablouei, S. Werner, K. Dogancay, Y. Huang, Analysis of a reduced-communication diffusion LMS algorithm. Signal Process. 117, 355–361 (2015)

    Article  Google Scholar 

  5. 5.

    S. Ashkezari-Toussi, H. Sadoghi-Yazdi, Robust diffusion LMS over adaptive networks. Signal Process. 158, 201–209 (2019)

    Article  Google Scholar 

  6. 6.

    F.S. Cattivelli, A.H. Sayed, Diffusion LMS strategies for distributed estimation. IEEE Trans. Signal Proc. 58, 1035–1048 (2010)

    MathSciNet  Article  Google Scholar 

  7. 7.

    H. Chang, W. Li, Correction-based diffusion LMS algorithms for secure distributed estimation under attacks. Digital Signal Process. 102, 102735 (2020)

    Article  Google Scholar 

  8. 8.

    H. Chang, W. Li, Correction-based diffusion LMS algorithms for distributed estimation. Circuit Syst. Signal Process. 39, 4136–4154 (2020)

    Article  Google Scholar 

  9. 9.

    F. Chen, X. Shao, Broken-motifs diffusion LMS algorithm for reducing communication load. Signal Process. 133, 213–218 (2017)

    Article  Google Scholar 

  10. 10.

    F. Chen, S. Deng, Y. Hua, S. Duan, L. Wang, J. Wu, Communication-reducing algorithm of distributed least mean square algorithm with neighbor-partial diffusion. Circuit Syst. Signal Process. 39, 4416–4435 (2020)

    Article  Google Scholar 

  11. 11.

    S. Chouvardas, K. Slavakis, S. Theodoridis, Trading off complexity with communication costs in distributed adaptive learning via Krylov subspaces for dimensionality reduction. IEEE J. Sel. Top. Signal Process. 7(2), 257–273 (2013)

    Article  Google Scholar 

  12. 12.

    E. Harrane, R. Flamary, C. Richard, On reducing the communication cost of the diffusion LMS algorithm. IEEE Trans. Signal Inf. Process. Over Netw. 5(1), 100–112 (2019)

    MathSciNet  Article  Google Scholar 

  13. 13.

    L. Hu, F. Chen, S. Duan, L. Wang, J. Wu, An improved diffusion affine projection estimation algorithm for wireless sensor networks. Circuit Syst. Signal Process. 39, 3173–3188 (2020)

    Article  Google Scholar 

  14. 14.

    W. Huang, C. Chen, X. Yao, Q. Li, Diffusion fused sparse LMS algorithm over networks. Signal Process. 171, 107497 (2020)

    Article  Google Scholar 

  15. 15.

    A. Javaheri, H. Zayyani, M.A.T. Figueiredo, F. Marvasti, Robust sparse recovery in impulsive noise via continuous mixed norm. IEEE Signal Process. Lett. 25(8), 1146–1150 (2018)

    Article  Google Scholar 

  16. 16.

    M. Korki, H. Zayyani, Weighted diffusion continuous mixed p-norm algorithm for distributed estimation in non-uniform noise environment. Signal Process. 164, 225–233 (2019)

    Article  Google Scholar 

  17. 17.

    J.W. Lee, S.E. Kim, W.J. Song, Data-selective diffusion LMS for reducing communication overhead. Signal Process. 113, 211–217 (2015)

    Article  Google Scholar 

  18. 18.

    J.W. Lee, J.T. Kong, W.J. Song, S.E. Kim, Data-reserved periodic diffusion LMS with low communication cost over networks. IEEE Access 6, 54636–54650 (2018)

    Article  Google Scholar 

  19. 19.

    C.G. Lopes, A.H. Sayed, Diffusion least-mean squares over adaptive networks: formulation and performance analysis. IEEE Trans. Signal Proc. 56, 3122–3136 (2008)

    MathSciNet  Article  Google Scholar 

  20. 20.

    J. Ni, J. Chen, X. Chen, Diffusion sign-error LMS algorithm: formulation and stochastic behavior analysis. Signal Process. 128, 142–149 (2016)

    Article  Google Scholar 

  21. 21.

    A. Rastegarnia, Reduced-communication diffusion RLS for distributed estimation over multi-agent networks. IEEE Trans. Circuit Syst. Part II Express Briefs 67(1), 177–181 (2020)

    Article  Google Scholar 

  22. 22.

    A. H. Sayed, Adaptation, Learning and Optimization Over Networks, Foundations and Trends in Machine Learning (2014)

  23. 23.

    M.O. Sayin, S.S. Kozat, Compressive diffusion strategies over distributed networks for reduced communication load. IEEE Trans. Signal Process. 62(20), 5308–5323 (2014)

    MathSciNet  Article  Google Scholar 

  24. 24.

    M.O. Sayin, S.S. Kozat, Single bit and reduced dimension diffusion strategies over distributed networks. IEEE Signal Process. Lett. 20(10), 976–979 (2013)

    Article  Google Scholar 

  25. 25.

    H. Shiri, M.A. Tinati, M. Coudreanu, G. Azarnia, Distributed sparse diffusion estimation with reduced communication cost. IET Signal Process. 12(8), 1043–1052 (2018)

    Article  Google Scholar 

  26. 26.

    A. Weron, R. Weron, Computer simulation of Levy alpha-stable variables and processes. Lect. Notes Phys. 457(6), 379–392 (1995)

    MathSciNet  Article  Google Scholar 

  27. 27.

    H. Zayyani, Robust minimum disturbance diffusion LMS for distributed estimation. IEEE Trans. Circuit Syst. Part II Express Briefs Early Access (2020). https://doi.org/10.1109/TCSII.2020.3004507

    Article  Google Scholar 

  28. 28.

    H. Zayyani, M. Korki, F. Marvasti, Bayesian hypothesis testing detector for one bit diffusion LMS with blind missing samples. Signal Process. 146, 61–65 (2018)

    Article  Google Scholar 

  29. 29.

    H. Zayyani, M. Korki, F. Marvasti, A distributed 1-bit compressed sensing algorithm robust to impulsive noise. IEEE Commun. Lett. 20(6), 1132–1135 (2016)

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hadi Zayyani.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Appendices

Appendix A: Optimizing the Coefficients of \(h_r\)

The disturbance incurred by the combination step is as

$$\begin{aligned} D=||\varvec{\omega }_{k,i+1}-\varvec{\omega }_{k,i}||^2_2=\left| \left| \sum _{l\in \mathcal N_\mathrm {k}}c_{lk}\tilde{{{\varvec{\varphi }}}}_{l,i+1}-\varvec{\omega }_{k,i}\right| \right| ^2_2. \end{aligned}$$
(16)

In the following, for the brevity, we omit the index k, i and \(i+1\) in somewhere. Hence, we have

$$\begin{aligned} D= & {} \left| \left| \sum _{l=1}^Lc_{lk}\tilde{{{\varvec{\varphi }}}}_{l}+\sum _{l=L+1}^d c_{lk}{{\varvec{\varphi }}}_{l}-\varvec{\omega }_{k}\right| \right| ^2_2 \nonumber \\= & {} \left| \left| \sum _{l=1}^Lc_{lk}(\sum _{r=L+1}^d h_r{{\varvec{\varphi }}}_r)+\sum _{l=L+1}^d c_{lk}{{\varvec{\varphi }}}_{l}-\varvec{\omega }_{k}\right| \right| ^2_2. \end{aligned}$$
(17)

where \(h_{rk}\) is replaced by \(h_r\) for simplicity. Then, some simple manipulation results to the following formula for the disturbance:

$$\begin{aligned} D=\left| \left| \sum _{r=L+1}^d(h_r\beta _k+c_{lk}){{\varvec{\varphi }}}_r-\varvec{\omega }_{k}\right| \right| ^2_2, \end{aligned}$$
(18)

where \(\beta _k\triangleq \sum _{l=1}^L c_{lk}\). By expanding the above formula, we can write

$$\begin{aligned} D=\left( \sum _{r=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}_r-\varvec{\omega }_{k}\right) ^T\left( \sum _{r^{'}=L+1}^d(h_{r^{'}}\beta _k+c_{r^{'}k}){{\varvec{\varphi }}}_{r^{'}}-\varvec{\omega }_{k}\right) , \end{aligned}$$
(19)

which results to the sum of four terms as follows:

$$\begin{aligned} D= & {} \sum _{r=L+1}^d\sum _{r^{'}=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}^T_{r}{{\varvec{\varphi }}}_{r^{'}}(h_{r^{'}}\beta _k+c_{r^{'}k}) \nonumber \\&\quad -\, \varvec{\omega }_{k}^{T}\left( \sum _{r^{'}=L+1}^d(h_{r^{'}}\beta _k+c_{r^{'}k}){{\varvec{\varphi }}}_{r^{'}}\right) -\left( \sum _{r=L+1}^d(h_r\beta _k+c_{rk}){{\varvec{\varphi }}}^T_{r}\right) \varvec{\omega }_k+||\varvec{\omega }_k||_2^2.\nonumber \\ \end{aligned}$$
(20)

Since \(||\varvec{\omega }_k||_2^2\) is constant with respect to the coefficients \(h_r\), minimizing the disturbance is equivalent to minimizing the following expression:

$$\begin{aligned} \tilde{D}= & {} \sum _{r=L+1}^d\sum _{r^{'}=L+1}^d(h_r\beta _k+c_{rk})(h_{r^{'}}\beta _k+c_{r^{'}k})\gamma _{r,r^{'}} \nonumber \\&\quad -\, 2\varvec{\omega }_{k}^{T}\Big (\sum _{r=L+1}^d(h_{r}\beta _k+c_{rk}){{\varvec{\varphi }}}_{r}\Big ), \end{aligned}$$
(21)

where \(\gamma _{r,r^{'}}\triangleq {{\varvec{\varphi }}}^T_{r}{{\varvec{\varphi }}}_{r^{'}}\). To minimize \(\tilde{D}\) with respect to \(h_r\), taking the partial derivative and make this equal to zero. So, with \(\frac{\partial \tilde{D}}{\partial h_p}=0\), we reach to the following linear equations:

$$\begin{aligned} \sum _{r=L+1}^d2\beta _k(h_{r}\beta _k+c_{rk})\gamma _{p,r}+2(h_{p}\beta _k+c_{pk})\beta _k\gamma _{r,r}-2\varvec{\omega }_k^T\beta _k{{\varvec{\varphi }}}_p=0. \end{aligned}$$
(22)

The above linear equations with respect to the coefficients, \(h_p\) for \(L_k+1\le p\le d_k\), can be written in a matrix format as in (7).

Appendix B: Calculating the Necessary and Sufficient Condition of Stability

If we define the disturbance difference as \(\Delta \triangleq D_{i+1}-D_i\), then it would be equal to

$$\begin{aligned} \Delta =\tilde{\varvec{\omega }}^T_{k,i+1}\mathbf {B}_{i+1}\tilde{\varvec{\omega }}_{k,i+1}-\tilde{\varvec{\omega }}^T_{k,i}\mathbf {B}_i\tilde{\varvec{\omega }}_{k,i}. \end{aligned}$$
(23)

To guarantee the stability of the proposed algorithm, the disturbance difference should be non-positive \(\Delta \le 0\). To ensure this, we expand and simplify the expression of \(\Delta \). To do so, we write

$$\begin{aligned}&\tilde{\varvec{\omega }}_{k,i+1}=\varvec{\omega }_{k,i+1}-\sum _{l}c_{lk}{{\varvec{\varphi }}}_{l,i+1}, \end{aligned}$$
(24)
$$\begin{aligned}&\tilde{\varvec{\omega }}_{k,i}=\varvec{\omega }_{k,i}-\sum _{l}c_{lk}{{\varvec{\varphi }}}_{l,i}. \end{aligned}$$
(25)

If we compute the difference of (24) and (25), with some simple simplification, we have

$$\begin{aligned} \tilde{\varvec{\omega }}_{k,i+1}-\tilde{\varvec{\omega }}_{k,i}=-{{\varvec{\varphi }}}_{k,i+1}+\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}. \end{aligned}$$
(26)

Therefore, we have \(\tilde{\varvec{\omega }}_{k,i+1}=\tilde{\varvec{\omega }}_{k,i}-{{\varvec{\varphi }}}_{k,i+1}+\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}\). Defining \(\mathbf {f}_{k,i}\triangleq {{\varvec{\varphi }}}_{k,i+1}-\sum _lc_{lk}{{\varvec{\varphi }}}_{l,i}\), we have \(\tilde{\varvec{\omega }}_{k,i+1}=\tilde{\varvec{\omega }}_{k,i}-\mathbf {f}_{k,i}\). Replacing this into (23), we have

$$\begin{aligned} \Delta =(\tilde{\varvec{\omega }}_{k,i+1}-\mathbf {f}_{k,i})^T\mathbf {B}_{i+1}(\tilde{\varvec{\omega }}_{k,i+1}-\mathbf {f}_{k,i})-\tilde{\varvec{\omega }}^T_{k,i}\mathbf {B}_i\tilde{\varvec{\omega }}_{k,i}. \end{aligned}$$
(27)

By some simple manipulations in the above formula, we reach to the following quadratic expression:

$$\begin{aligned} \Delta =\tilde{\varvec{\omega }}_{k,i+1}\mathbf{Q }_i\tilde{\varvec{\omega }}_{k,i+1}-2\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}\tilde{\varvec{\omega }}_{k,i+1}+\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}\mathbf {f}_{k,i}, \end{aligned}$$
(28)

where \(\mathbf{Q }_i\triangleq \mathbf {B}_{i+1}-\mathbf {B}_i=\varvec{\Phi }_i(\varvec{\Phi }_i^T\varvec{\Phi }_i)^{-1}\varvec{\Phi }_i^T-\varvec{\Phi }_{i+1}(\varvec{\Phi }_{i+1}^T\varvec{\Phi }_{i+1})^{-1}\varvec{\Phi }_{i+1}^T\). To be ensure that this difference is non-positive, a necessary condition is that the matrix \(\mathbf{Q }_i\le 0\) be non-positive. In other words, the necessary condition for stability is that \(\varvec{\Phi }_{i+1}(\varvec{\Phi }_{i+1}^T\varvec{\Phi }_{i+1})^{-1}\varvec{\Phi }_{i+1}^T-\varvec{\Phi }_i(\varvec{\Phi }_i^T\varvec{\Phi }_i)^{-1}\varvec{\Phi }_i^T\ge 0\) is nonnegative definite. To find the necessary and sufficient condition for stability, in addition to the necessary condition \(\mathbf{Q }_i\le 0\), the maximum of the quadratic expression in (28) should be non-positive. So, taking the derivative of \(\Delta \) and enforcing it to zero \(\frac{\partial \Delta }{\partial \tilde{\varvec{\omega }}_{k,i}}=0\), we have the maximizing vector equal to \(\tilde{\varvec{\omega }}_{k,i}=\mathbf{Q }^{-1}_i\mathbf {B}_{i+1}^T\mathbf {f}_{k,i}\). Replacing this into (28), and with some manipulations, we reach

$$\begin{aligned} \Delta _{\mathrm {max}}=\mathbf {f}^T_{k,i}\mathbf {B}_{i+1}(\mathbf{I }-\mathbf{Q }^{-1}_i\mathbf {B}^T_{i+1})\mathbf {f}_{k,i}\le 0. \end{aligned}$$
(29)

So, we should have \(\mathbf {R}_i\triangleq \mathbf {B}_{i+1}(\mathbf{I }-\mathbf{Q }^{-1}_i\mathbf {B}^T_{i+1})\le 0\). Hence, the necessary and sufficient condition for stability of the proposed algorithm is that at every iteration, we have?

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zayyani, H. Communication Reducing Diffusion LMS Robust to Impulsive Noise Using Smart Selection of Communication Nodes. Circuits Syst Signal Process (2021). https://doi.org/10.1007/s00034-021-01840-4

Download citation

Keywords

  • Distributed estimation
  • Diffusion LMS
  • minimum disturbance
  • Communication cost
  • Impulsive noise