Circuits, Systems, and Signal Processing

, Volume 38, Issue 4, pp 1736–1750 | Cite as

Diffusion Sparse Sign Algorithm with Variable Step-Size

  • Feng Chen
  • Xiaoxia Liu
  • Shukai DuanEmail author
  • Lidan Wang
  • Jiagui Wu


In this paper, we propose the diffusion sparse sign algorithm with variable step-size for distributed estimation in sparse and impulsive interference environments. Firstly, we address the problem of in-network distributed estimation for sparse vectors under the impulsive noise environment. In order to exploit the sparsity of the vector of interest, we incorporate the sparse norms (\({l_1}\)-norm and \(RW{l_1}\)-norm) into the cost function of the standard diffusion sign algorithm, which accelerates the convergence speed of zero or near-zero components. In addition, we propose the adaptive variable step-size to further improve the convergence rate of the proposed algorithm. The variable step-size is derived by the correlation entropy, which contains a modified Gaussian kernel function and is robust to impulsive noise. In this paper, every node combines its correlation entropy function with the information of its neighborhood to drive the variable step-size at each iteration. Simulation results show that the proposed algorithm outperforms the standard diffusion SA in the sparse and impulsive system and the convergence rate of the proposed algorithm is faster than constant step-size algorithms.


Diffusion Sparsity Variable step-size Impulsive noise Sign algorithm 


  1. 1.
    R. Arablouei, S. Werner, K. Dogancay, Diffusion-based distributed adaptive estimation utilizing gradient-descent total least-squares, in IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE, 2013), pp. 5308–5312Google Scholar
  2. 2.
    A. Bertrand, M. Moonen, Consensus-based distributed total least squares estimation in ad hoc wireless sensor networks. IEEE Trans. Signal Process. 59(5), 2320–2330 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    J.A. Bazerque, G.B. Giannakis, Distributed spectrum sensing for cognitive radio networks by exploiting sparsity. IEEE Trans. Signal Process. 58(3), 1847–1862 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    E.J. Cands, M.B. Wakin, S.P. Boyd, Enhancing sparsity by reweighted \({l_1}\), minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    F.S. Cattivelli, A.H. Sayed, Diffusion LMS strategies for distributed estimation. IEEE Trans. Signal Process. 58(3), 1035–1048 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    F.S. Cattivelli, C.G. Lopes, A.H. Sayed, Diffusion strategies for distributed Kalman filtering: formulation and performance analysis, in Proceedings of the Iapr Workshop on Cognitive Information Processing (2008)Google Scholar
  7. 7.
    F.S. Cattivelli, A.H. Sayed, Diffusion strategies for distributed Kalman filtering and smoothing. IEEE Trans. Autom. Control 55(9), 2069–2084 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    F.S. Cattivelli, A.H. Sayed, Diffusion mechanisms for fixedpoint distributed Kalman smoothing, in Signal Processing Conference (2008)Google Scholar
  9. 9.
    T. Fan, Y. Lin, A variable step-size strategy based on error function for sparse system identification. Circuits Syst. Signal Process. 36, 1301–1310 (2017)CrossRefzbMATHGoogle Scholar
  10. 10.
    S. Huang, C. Li, Distributed sparse total least-squares over networks. IEEE Trans. Signal Process. 63(11), 2986–2998 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    C. Li, P. Shen, Y. Liu, Z. Zhang, Diffusion information theoretic learning for distributed estimation over network. IEEE Trans. Signal Process. 61(16), 4011–4024 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    C.G. Lopes, A.H. Sayed, Diffusion least-mean squares over adaptive networks: formulation and performance analysis. IEEE Trans. Signal Process. 56(7), 3122–3136 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    C.G. Lopes, A.H. Sayed, Distributed adaptive incremental strategies: formulation and performance analysis. IEEE Int. Conf. Acoust. Speech Signal Process. 3, 584–587 (2006)Google Scholar
  14. 14.
    M. Li, L. Li, H.M. Tai, Variable step size LMS algorithm based on function control. Circuits Syst. Signal Process. 32, 3121–3130 (2013)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Y. Liu, C. Li, Z. Zhang, Diffusion sparse least-mean squares over networks. IEEE Trans. Signal Process. 60(8), 4480–4485 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Z. Liu, Y. Liu, C. Li, Distributed sparse recursive least-squares over networks. IEEE Trans. Signal Process. 62(6), 1386–1395 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    K. Mayyas, A new variable step size control method for the transform domain LMS adaptive algorithm. Circuits Syst. Signal Process. 24(6), 703–721 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    R.K. Martin, W.A. Sethares, R.C. Williamson, C.R. Johnson Jr., Exploiting sparsity in adaptive filters. IEEE Trans. Signal Process. 50(8), 1883–1894 (2002)CrossRefGoogle Scholar
  19. 19.
    W. Ma, H. Qu, G. Gui, L. Xu, J. Zhao, B. chen, Maximum correntropy criterion based sparse adaptive filtering algorithms for robust channel estimation under non-Gaussian environments. J. Frankl. Inst. 352(7), 2708–2727 (2015)CrossRefzbMATHGoogle Scholar
  20. 20.
    J. Ni, J. Chen, Diffusion sign-error LMS algorithm: formulation and stochastic behavior analysis. Signal Process. 128, 142–149 (2016)CrossRefGoogle Scholar
  21. 21.
    S.S. Ram, A. Nedic, V.V. Veeravalli, in Asilomar Conference on stochastic incremental gradient descent for estimation in sensor networks (IEEE, 2007), pp. 582–586Google Scholar
  22. 22.
    A. Singh, J. Principe, Using correntropy as a cost function in linear adaptive filters, in International Joint Conference on Neural Networks. (IEEE, 2009), pp. 2950–2955Google Scholar
  23. 23.
    B.W. Silverman, Density Estimation for Statistics and Data Analysis (Chapman and Hall, London, 1986)CrossRefzbMATHGoogle Scholar
  24. 24.
    I.D. Schizas, G. Mateos, G.B. Giannakis, Distributed LMS for consensus-based in-network adaptive processing. IEEE Trans. Signal Process. 8(6), 2365–2381 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    L. Shi, H. Zhao, Variable step size norm-constrained adaptive filtering algorithms. Circuits Syst. Signal Process. 36(10), 1–14 (2017)zbMATHGoogle Scholar
  26. 26.
    L. Shi, H. Zhao, Two diffusion proportionate sign subband adaptive filtering algorithms. Circuits Syst. Signal Process. 36(10), 1–18 (2017)zbMATHGoogle Scholar
  27. 27.
    S.S. Stankovic, M.S. Stankovic, D.S. Stipanovic, Decentralized parameter estimation by consensus based stochastic approximation. IEEE Trans. Autom. Control 56(3), 531–543 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    T. Shao, Y.R. Zheng, J. Benesty, An affine projection sign algorithm robust against impulsive interferences. IEEE Signal Process. Lett. 17(4), 327–330 (2010)CrossRefGoogle Scholar
  29. 29.
    X. Shao, F. Chen, Q. Ye, S. Duan, A robust diffusion estimation algorithm with self-adjusting step-size in WSNs. Sensors 17(4), 824 (2017)CrossRefGoogle Scholar
  30. 30.
    W. Wang, J. Zhao, H. Qu, B. Chen, A correntropy inspired variable step-size sign algorithm against impulsive noises. Signal Process. 141, 168–175 (2017)CrossRefGoogle Scholar
  31. 31.
    W. Wang, H. Zhao, B. Chen, Robust adaptive volterra filter under maximum correntropy criteria in impulsive environments. Circuits Syst. Signal Process. 36, 4097–4117 (2017)CrossRefzbMATHGoogle Scholar
  32. 32.
    Z. Wu, S. Peng, B. Chen, H. Zhao, Robust hammerstein adaptive filtering under maximum correntropy criterion. Entropy 17(10), 7149–7166 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    H. Zhu, G. Leus, G.B. Giannakis, Sparsity-cognizant total least-squares for perturbed compressive sampling. IEEE Trans. Signal Process. 59(5), 2002–2016 (2011)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Feng Chen
    • 1
    • 2
    • 3
    • 4
  • Xiaoxia Liu
    • 1
    • 3
  • Shukai Duan
    • 1
    Email author
  • Lidan Wang
    • 1
  • Jiagui Wu
    • 1
    • 3
  1. 1.College of Electronic and Information EngineeringSouthwest UniversityChongqingChina
  2. 2.School of Mathematics and StatisticsSouthwest UniversityChongqingChina
  3. 3.Key Laboratory of Nonlinear Circuits and Intelligent Information ProcessingChongqingChina
  4. 4.Chongqing Collaborative Innovation Center for Brain ScienceChongqingChina

Personalised recommendations