Advertisement

Neural Computing and Applications

, Volume 31, Issue 4, pp 1007–1021 | Cite as

Distributed cooperative learning algorithms using wavelet neural network

  • Jin Xie
  • Weisheng Chen
  • Hao DaiEmail author
Original Article

Abstract

This paper investigates the distributed cooperative learning (DCL) problems over networks, where each node only has access to its own data generated by the unknown pattern (map or function) uniformly, and all nodes cooperatively learn the pattern by exchanging local information with their neighboring nodes. These problems cannot be solved by using traditional centralized algorithms. To solve these problems, two novel DCL algorithms using wavelet neural networks are proposed, including continuous-time DCL (CT-DCL) algorithm and discrete-time DCL (DT-DCL) algorithm. Combining the characteristics of neural networks with the properties of the wavelet approximation, the wavelet series are used to approximate the unknown pattern. The DCL algorithms are used to train the optimal weight coefficient matrix of wavelet series. Moreover, the convergence of the proposed algorithms is guaranteed by using the Lyapunov method. Compared with existing distributed optimization strategies such as distributed average consensus (DAC) and alternating direction method of multipliers (ADMM), our DT-DCL algorithm requires less information communications and training time than ADMM strategy. In addition, it achieves higher accuracy than DAC strategy when the network consists of large amounts of nodes. Moreover, the proposed CT-DCL algorithm using a proper step size is more accurate than the DT-DCL algorithm if the training time is not considered. Several illustrative examples are presented to show the efficiencies and advantages of the proposed algorithms.

Keywords

Distributed cooperative learning (DCL) Wavelet approximation Wavelet Neural Networks (WNNs) Lyapunov method 

Notes

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant Numbers: 61503292, 61673308, 61673014), which made it possible to undertake this research.

Compliance with ethical standards

Conflict of interest

We declare that we have no conflicts of interest.

References

  1. 1.
    Chen W, Hua S, Ge SS (2014) Consensus-based distributed cooperative learning control for a group of discrete-time nonlinear multi-agent systems using neural networks. Automatica 50(9):2254–2268MathSciNetzbMATHGoogle Scholar
  2. 2.
    Boyd S, Parikh N, Chu E et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1–122zbMATHGoogle Scholar
  3. 3.
    Li J, Lin X, Rui X et al (2014) A distributed approach toward discriminative distance metric learning. IEEE Trans Neural Netw Learn Syst 26(9):2111–2122MathSciNetGoogle Scholar
  4. 4.
    Tekin C, Van der SM (2015) Distributed online learning via cooperative contextual bandits. Signal Process IEEE Trans 63(14):3700–3714MathSciNetzbMATHGoogle Scholar
  5. 5.
    Chen T, Wang C, Hill DJ (2014) Rapid oscillation fault detection and isolation for distributed systems via deterministic learning. IEEE Trans Neural Netw Learn Syst 25(6):1187–1199Google Scholar
  6. 6.
    Mertikopoulos P, Belmega EV, Moustakas AL et al (2012) Distributed learning policies for power allocation in multiple access channels. Onomr 30(1):96–106Google Scholar
  7. 7.
    Predd JB, Kulkarni SR, Vincent PH (2005) Distributed learning in wireless sensor networks. IEEE Signal Process Mag 23(4):56–69Google Scholar
  8. 8.
    Scardapane S, Wang D, Panella M (2016) A decentralized training algorithm for echo state networks in distributed big data applications. Neural Netw 78:65–74Google Scholar
  9. 9.
    Georgopoulos L, Hasler M (2014) Distributed machine learning in networks by consensus. Neurocomputing 124(2):2–12Google Scholar
  10. 10.
    Chen J, Sayed AH (2011) Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Trans Signal Process 60(8):4289–4305MathSciNetzbMATHGoogle Scholar
  11. 11.
    Chen W, Hua S, Zhang H (2015) Consensus-based distributed cooperative learning from closed-loop neural control systems. IEEE Trans Neural Netw Learn Syst 26(2):331–345MathSciNetGoogle Scholar
  12. 12.
    Ai W, Chen W, Xie J (2016) A zero-gradient-sum algorithm for distributed cooperative learning using a feedforward neural network with random weights. Inf Sci 373:404–418Google Scholar
  13. 13.
    Ai W, Chen W, Xie J (2016) Distributed learning for feedforward neural networks with random weights using an event-triggered communication scheme. Neurocomputing 224:184–194Google Scholar
  14. 14.
    Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. Proc IEEE 95(1):215–233zbMATHGoogle Scholar
  15. 15.
    Xiao L, Boyd S, Kim SJ (2007) Distributed average consensus with least-mean-square deviation. J Parallel Distrib Comput 67(1):33–46zbMATHGoogle Scholar
  16. 16.
    Aysal T, Coates M, Rabbat M (2008) Distributed average consensus with dithered quantization. IEEE Trans Signal Process 56(10):4905–4918MathSciNetzbMATHGoogle Scholar
  17. 17.
    Lim C, Lee S, Choi JH et al (2014) Efficient implementation of statistical model-based voice activity detection using Taylor series approximation. IEICE Trans Fundam Electron Commun Comput Sci 97(3):865–868Google Scholar
  18. 18.
    Sharapudinov II (2014) Approximation of functions in variable-exponent Lebesgue and Sobolev spaces by finite Fourier–Haar series. Russ Acad Sci Sb Math 205(205):145–160MathSciNetzbMATHGoogle Scholar
  19. 19.
    Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67Google Scholar
  20. 20.
    Yang C, Jiang K et al (2017) Neural control of bimanual robots with guaranteed global stability and motion precision. IEEE Trans Ind Inf 13(3):1162–1171Google Scholar
  21. 21.
    Yang C, Wang X et al (2017) Neural-learning based telerobot control with guaranteed performance. IEEE Trans Cybern (in press). doi: 10.1109/TCYB.2016.2573837
  22. 22.
    Cui R, Yang C et al (2017) Adaptive neural network control of AUVs with control input nonlinearities using reinforcement learning. IEEE Trans Syst Man Cybern Syst 47(6):1019–1029Google Scholar
  23. 23.
    Wu S, Er MJ (2000) Dynamic fuzzy neural networks-a novel approach to function approximation. IEEE Trans Syst Man Cybern Part B Cybern A Publ IEEE Syst Man Cybern Soc 30(2):358–364Google Scholar
  24. 24.
    Ferrari S, Stengel RF (2005) Smooth function approximation using neural networks. IEEE Trans Neural Netw 16(1):24–38Google Scholar
  25. 25.
    Yang C, Yi Z, Zuo L (2008) Function approximation based on twin support vector machines. In: IEEE conference on cybernetics and intelligent systems, pp 259–264Google Scholar
  26. 26.
    Zhang Q, Benveniste A (1991) Approximation by nonlinear wavelet networks. In: International conference on acoustics, speech and signal processing. ICASSP-91. pp 3417–3420Google Scholar
  27. 27.
    Zhang Q, Benveniste A (1992) Wavelet networks. IEEE Trans Neural Netw 3(6):889Google Scholar
  28. 28.
    Delyon B, Juditsky A, Benveniste A (1995) Accuracy analysis for wavelet approximations. IEEE Trans Neural Netw 6(2):332–348Google Scholar
  29. 29.
    Zainuddin Z, Pauline O (2011) Modified wavelet neural network in function approximation and its application in prediction of time-series pollution data. Appl Soft Comput 11(8):4866–4874Google Scholar
  30. 30.
    Zainuddin Z, Ong P (2013) Design of wavelet neural networks based on symmetry fuzzy C-means for function approximation. Neural Comput Appl 23(1):247–259Google Scholar
  31. 31.
    Zainuddin Z, Ong P (2016) Optimization of wavelet neural networks with the firefly algorithm for approximation problems. Neural Comput Appl 28(7):1715–1728Google Scholar
  32. 32.
    Hou M, Han X, Gan Y (2009) Constructive approximation to real function by wavelet neural networks. Neural Comput Appl 18(8):883–889Google Scholar
  33. 33.
    Oysal Y, Yilmaz S (2010) An adaptive wavelet network for function learning. Neural Comput Appl 19(3):383–392Google Scholar
  34. 34.
    Xu J, Yan R (2011) Adaptive learning control for finite interval tracking based on constructive function approximation and wavelet. IEEE Trans Neural Netw 22(6):893–905Google Scholar
  35. 35.
    Alexandridis AK, Zapranis AD (2013) Wavelet neural networks: a practical guide. Neural Netw 42(1):1–27zbMATHGoogle Scholar
  36. 36.
    Yang C, Wang X et al (2016) Teleoperation control based on combination of wave variable and neural networks. IEEE Trans Syst Man Cybern Syst (in press). doi: 10.1109/TSMC.2016.2615061
  37. 37.
    Chen S, Zhao H, Zhang S et al (2013) Study of ultra-wideband fuze signal processing method based on wavelet transform. IET Radar Sonar Navig 8(3):167–172Google Scholar
  38. 38.
    Courroux S, Chevobbe S, Darouich M et al (2013) Use of wavelet for image processing in smart cameras with low hardware resources. J Syst Archit 59(10):826–832Google Scholar
  39. 39.
    Pavez E, Silva JF (2012) Analysis and design of wavelet-packet cepstral coefficients for automatic speech recognition. Speech Commun 54(6):814–835Google Scholar
  40. 40.
    Siddiqi MH, Lee SW, Khan AM (2014) Weed image classification using wavelet transform, stepwise linear discriminant analysis and support vector machines for an automatic spray control system. J Inf Sci Eng 30(4):1227–1244Google Scholar
  41. 41.
    Yan R, Gao RX, Chen X (2014) Wavelets for fault diagnosis of rotary machines: a review with applications. Signal Process 96(5):1–15Google Scholar
  42. 42.
    Ganjefar S, Tofighi M (2015) Single-hidden-layer fuzzy recurrent wavelet neural network: applications to function approximation and system identification. Inf Sci 294:269–285MathSciNetzbMATHGoogle Scholar
  43. 43.
    Bazaraa MS, Goode JJ (1973) On symmetric duality in nonlinear programming. Oper Res 21(21):1–9MathSciNetzbMATHGoogle Scholar
  44. 44.
    Lu J, Tang CY (2011) Zero-gradient-sum algorithms for distributed convex optimization: the continuous-time case. IEEE Trans Autom Control 57(9):5474–5479MathSciNetGoogle Scholar

Copyright information

© The Natural Computing Applications Forum 2017

Authors and Affiliations

  1. 1.School of Aerospace Science and TechnologyXidian UniversityXi’anChina
  2. 2.School of Mathematics and StatisticsXidian UniversityXi’anChina

Personalised recommendations