A Comparison of Consensus Strategies for Distributed Learning of Random Vector Functional-Link Networks

  • Roberto Fierimonte
  • Simone Scardapane
  • Massimo Panella
  • Aurelio Uncini
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 54)

Abstract

Distributed machine learning is the problem of inferring a desired relation when the training data is distributed throughout a network of agents (e.g. robots in a robot swarm). Multiple families of distributed learning algorithms are based on the decentralized average consensus (DAC) protocol, an efficient algorithm for computing an average starting from local measurement vectors. The performance of DAC, however, is strongly dependent on the choice of a weighting matrix associated to the network. In this paper, we perform a comparative analysis of the relative performance of 4 different strategies for choosing the weighting matrix. As an applicative example, we consider the distributed sequential algorithm for Random Vector Functional-Link networks. As expected, our experimental simulations show that the training time required by the algorithm is drastically reduced when considering a proper initialization of the weights.

Keywords

Consensus Distributed machine learning Random vector functional-link 

References

  1. 1.
    Baccarelli, E., Cordeschi, N., Mei, A., Panella, M., Shojafar, M., Stefa, J.: Energy-efficient dynamic traffic offloading and reconfiguration of networked datacenters for big data stream mobile computing: review, challenges, and a case study. IEEE Netw. Mag. (2015)Google Scholar
  2. 2.
    Barbarossa, S., Sardellitti, S., Di Lorenzo, P.: Distributed detection and estimation in wireless sensor networks. In: Chellapa, R., Theodoridis, S. (eds.) E-Reference Signal Processing, pp. 329–408. Elsevier (2013)Google Scholar
  3. 3.
    Comminiello, D., Scarpiniti, M., Azpicueta-Ruiz, L., Arenas-Garcia, J., Uncini, A.: Functional link adaptive filters for nonlinear acoustic echo cancellation. IEEE Trans. Audio, Speech, Lang. Process. 21(7), 1502–1512 (2013)CrossRefGoogle Scholar
  4. 4.
    Forero, P.A., Cano, A., Giannakis, G.B.: Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11, 1663–1707 (2010)MathSciNetMATHGoogle Scholar
  5. 5.
    Georgopoulos, L., Hasler, M.: Distributed machine learning in networks by consensus. Neurocomputing 124, 2–12 (2014)CrossRefGoogle Scholar
  6. 6.
    Olfati-Saber, R., Fax, J.A., Murray, R.M.: Consensus and cooperation in networked multi-agent systems. Proc. IEEE 95(1), 215–233 (2007)CrossRefGoogle Scholar
  7. 7.
    Olfati-Saber, R., Murray, R.M.: Consensus problems in networks of agents with switching topology and time-delays. IEEE Trans. Autom. Control 49(9), 1520–1533 (2004)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Pao, Y.H., Park, G.H., Sobajic, D.J.: Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6(2), 163–180 (1994)CrossRefGoogle Scholar
  9. 9.
    Scardapane, S., Comminiello, D., Scarpiniti, M., Uncini, A.: Online sequential extreme learning machine with kernels. IEEE Trans. Neural Netw. Learn. Syst. (2015)Google Scholar
  10. 10.
    Scardapane, S., Fierimonte, R., Wang, D., Panella, M., Uncini, A.: Distributed music classification using random vector functional-link nets. In: Accepted for presentation at 2015 IEEE/INNS International Joint Conference on Neural Networks (IJCNN’15) (2015)Google Scholar
  11. 11.
    Scardapene, S., Wang, D., Panella, M., Uncini, A.: Distributed learning for random vector functional-link networks. Inf. Sci. 301, 271–284 (2015)Google Scholar
  12. 12.
    Xiao, L., Boyd, S.: Fast linear iterations for distributed averaging. Syst. Control Lett. 53(1), 65–78 (2004)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Roberto Fierimonte
    • 1
  • Simone Scardapane
    • 1
  • Massimo Panella
    • 1
  • Aurelio Uncini
    • 1
  1. 1.Department of Information Engineering Electronics and Telecommunications (DIET)“Sapienza” University of RomeRomeItaly

Personalised recommendations