A Comparison of Consensus Strategies for Distributed Learning of Random Vector Functional-Link Networks
Distributed machine learning is the problem of inferring a desired relation when the training data is distributed throughout a network of agents (e.g. robots in a robot swarm). Multiple families of distributed learning algorithms are based on the decentralized average consensus (DAC) protocol, an efficient algorithm for computing an average starting from local measurement vectors. The performance of DAC, however, is strongly dependent on the choice of a weighting matrix associated to the network. In this paper, we perform a comparative analysis of the relative performance of 4 different strategies for choosing the weighting matrix. As an applicative example, we consider the distributed sequential algorithm for Random Vector Functional-Link networks. As expected, our experimental simulations show that the training time required by the algorithm is drastically reduced when considering a proper initialization of the weights.
KeywordsConsensus Distributed machine learning Random vector functional-link
- 1.Baccarelli, E., Cordeschi, N., Mei, A., Panella, M., Shojafar, M., Stefa, J.: Energy-efficient dynamic traffic offloading and reconfiguration of networked datacenters for big data stream mobile computing: review, challenges, and a case study. IEEE Netw. Mag. (2015)Google Scholar
- 2.Barbarossa, S., Sardellitti, S., Di Lorenzo, P.: Distributed detection and estimation in wireless sensor networks. In: Chellapa, R., Theodoridis, S. (eds.) E-Reference Signal Processing, pp. 329–408. Elsevier (2013)Google Scholar
- 9.Scardapane, S., Comminiello, D., Scarpiniti, M., Uncini, A.: Online sequential extreme learning machine with kernels. IEEE Trans. Neural Netw. Learn. Syst. (2015)Google Scholar
- 10.Scardapane, S., Fierimonte, R., Wang, D., Panella, M., Uncini, A.: Distributed music classification using random vector functional-link nets. In: Accepted for presentation at 2015 IEEE/INNS International Joint Conference on Neural Networks (IJCNN’15) (2015)Google Scholar
- 11.Scardapene, S., Wang, D., Panella, M., Uncini, A.: Distributed learning for random vector functional-link networks. Inf. Sci. 301, 271–284 (2015)Google Scholar