Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization
We consider a distributed multi-agent network system where the goal is to minimize a sum of convex objective functions of the agents subject to a common convex constraint set. Each agent maintains an iterate sequence and communicates the iterates to its neighbors. Then, each agent combines weighted averages of the received iterates with its own iterate, and adjusts the iterate by using subgradient information (known with stochastic errors) of its own function and by projecting onto the constraint set.
The goal of this paper is to explore the effects of stochastic subgradient errors on the convergence of the algorithm. We first consider the behavior of the algorithm in mean, and then the convergence with probability 1 and in mean square. We consider general stochastic errors that have uniformly bounded second moments and obtain bounds on the limiting performance of the algorithm in mean for diminishing and non-diminishing stepsizes. When the means of the errors diminish, we prove that there is mean consensus between the agents and mean convergence to the optimum function value for diminishing stepsizes. When the mean errors diminish sufficiently fast, we strengthen the results to consensus and convergence of the iterates to an optimal solution with probability 1 and in mean square.
KeywordsDistributed algorithm Convex optimization Subgradient methods Stochastic approximation
Unable to display preview. Download preview PDF.
- 2.Nedić, A., Bertsekas, D.P.: Convergence rate of incremental algorithms. In: Uryasev, S., Pardalos, P.M. (eds.) Stochastic Optimization: Algorithms and Applications, pp. 223–264. Kluwer Academic, Dordrecht (2001) Google Scholar
- 5.Nedić, A., Ozdaglar, A.: On the rate of convergence of distributed asynchronous subgradient methods for multi-agent optimization. In: Proceedings of the 46th IEEE Conference on Decision and Control, pp. 4711–4716 (2007) Google Scholar
- 7.Nedić, A., Ozdaglar, A., Parrilo, P.A.: Constrained consensus and optimization in multi-agent networks. Lab. for Information and Decision Systems Technical Report 2779, Massachusetts Institute of Technology (2008) Google Scholar
- 8.Rabbat, M.G., Nowak, R.D.: Distributed optimization in sensor networks. In: Proceedings of International Symposium on Information Processing in Sensor Networks, pp. 20–27 (2004) Google Scholar
- 10.Johansson, B., Rabi, M., Johansson, M.: A simple peer-to-peer algorithm for distributed optimization in sensor networks. In: Proceedings of the 46th IEEE Conference on Decision and Control, pp. 4705–4710 (2007) Google Scholar
- 11.Johansson, B.: On distributed optimization in networked systems. Ph.D. thesis, Royal Institute of Technology, Stockholm (2008) Google Scholar
- 12.Sundhar, R.S.: Distributed optimization in multi-agent systems with applications to distributed regression. Ph.D. thesis, University of Illinois at Urbana-Champaign (2009) Google Scholar
- 13.Nedić, A., Ozdaglar, A.: Cooperative distributed multi-agent optimization. In: Daniel, P.P., Eldar, Y.C. (eds.) Convex Optimization in Signal Processing and Communications, pp. 340–386. Cambridge University Press, Cambridge (2010) Google Scholar
- 20.Tsitsiklis, J.N.: Problems in decentralized decision making and computation. Ph.D. thesis, Massachusetts Institute of Technology (1984) Google Scholar
- 22.Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific, Belmont (1997) Google Scholar
- 23.Jadbabaie, A., Lin, J., Morse, S.: Coordination of groups of mobile autonomous agents using nearest neighbor rules. IEEE Trans. Automat. Control 48, 998–1001 (2003) Google Scholar
- 24.Spanos, D.S., Olfati-Saber, R., Murray, R.M.: Approximate distributed Kalman filtering in sensor networks with quantifiable performance. In: Proceedings of IEEE International Conference on Information Processing in Sensor Networks, pp. 133–139 (2005) Google Scholar
- 27.Lobel, I., Ozdaglar, A.: Distributed subgradient methods over random networks. Lab. for Information and Decision Systems Technical Report 2800, Massachusetts Institute of Technology (2008) Google Scholar
- 29.Nedić, A., Olshevsky, A., Ozdaglar, A., Tsitsiklis, J.N.: Distributed subgradient algorithms and quantization effects. In: Proceedings of the 47th IEEE Conference on Decision and Control, pp. 4177–4184 (2008) Google Scholar
- 30.Huang, M., Manton, J.H.: Stochastic approximation for consensus seeking: mean square and almost sure convergence. In: Proceedings of the 46th IEEE Conference on Decision and Control, pp. 306–311 (2007) Google Scholar
- 31.Ermoliev, Y.: Stochastic Programming Methods. Nauka, Moscow (1976) Google Scholar
- 33.Ermoliev, Y.: Stochastic quazigradient methods. In: Ermoliev, Y., Wets, R.J.-B. (eds.) Numerical Techniques for Stochastic Optimization, pp. 141–186. Springer, New York (1988) Google Scholar
- 34.Sundhar, R.S., Veeravalli, V.V., Nedić, A.: Distributed and non-autonomous power control through distributed convex optimization. In: Proceedings of the 28th IEEE Conference on Computer Communications INFOCOM, pp. 3001–3005 (2009) Google Scholar
- 37.Polyak, B.T.: Introduction to Optimization. Optimization Software, New York (1987) Google Scholar