Abstract
Decentralized optimization is a common paradigm used in distributed signal processing and sensing as well as privacy-preserving and large-scale machine learning. It is assumed that several computational entities locally hold objective functions and are connected by a network. The agents aim to commonly minimize the sum of the local objectives subject by making gradient updates and exchanging information with their immediate neighbors. Theory of decentralized optimization is pretty well-developed in the literature. In particular, it includes lower bounds and optimal algorithms. In this paper, we assume that along with an objective, each node also holds affine constraints. We discuss several primal and dual approaches to decentralized optimization problem with affine constraints.
The work of D. Yarmoshik in Sects. 1, 6, 7 was supported by the program “Leading Scientific Schools” (grant no. NSh-775.2022.1.1). The work of A. Rogozin and A. Gasnikov in Sects. 2–5 was supported by Russian Science Foundation (project No. 21-71- 30005).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Source code: https://github.com/niquepolice/decentr_constr_dual.
References
Erseghe, T.: Distributed optimal power flow using ADMM. IEEE Trans. Power Syst. 29(5), 2370–2380 (2014). https://doi.org/10.1109/TPWRS.2014.2306495
Falsone, A., Margellos, K., Garatti, S., Prandini, M.: Dual decomposition for multi-agent distributed optimization with coupling constraints. Automatica 84, 149–158 (2017)
Falsone, A., Notarnicola, I., Notarstefano, G., Prandini, M.: Tracking-ADMM for distributed constraint-coupled optimization. Automatica 117, 1–13 (2020)
Gorbunov, E., Dvinskikh, D., Gasnikov, A.: Optimal decentralized distributed algorithms for stochastic convex optimization. arXiv:1911.07363 (2019)
Harris, C.R., et al.: Array programming with NumPy. Nature 585(7825), 357–362 (2020). https://doi.org/10.1038/s41586-020-2649-2
Kakade, S.M., Shalev-Shwartz, S., Tewari, A.: On the duality of strong convexity and strong smoothness: learning applications and matrix regularization. Technical report. Toyota Technological Institute (2009)
Kovalev, D., Gasnikov, A., Richtárik, P.: Accelerated primal-dual gradient method for smooth and convex-concave saddle-point problems with bilinear coupling. arXiv preprint arXiv:2112.15199 (2021)
Kovalev, D., Salim, A., Richtárik, P.: Optimal and practical algorithms for smooth and strongly convex decentralized optimization. Adv. Neural. Inf. Process. Syst. 33, 18342–18352 (2020)
Liang, S., Wang, L.Y., Yin, G.: Distributed smooth convex optimization with coupled constraints. IEEE Trans. Autom. Control 65, 347–353 (2020)
Liang, S., Zheng, X., Hong, Y.: Distributed ninsmooth optimization with coupled inequality constraints via modified Lagrangian function. IEEE Trans. Autom. Control 63, 1753–1759 (2018)
Molzahn, D.K., et al.: A survey of distributed optimization and control algorithms for electric power systems. IEEE Trans. Smart Grid 8(6), 2941–2962 (2017). https://doi.org/10.1109/TSG.2017.2720471
Necoara, I., Nedelcu, V.: Distributed dual gradient methods and error bound conditions. arXiv:1401.4398 (2014)
Necoara, I., Nedelcu, V.: On linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems. Automatica 55, 209–216 (2015). https://doi.org/10.1016/j.automatica.2015.02.038, https://www.sciencedirect.com/science/article/pii/S0005109815001004
Necoara, I., Nedelcu, V., Dumitrache, I.: Parallel and distributed optimization methods for estimation and control in networks. J. Process Control 21(5), 756–766 (2011). https://doi.org/10.1016/j.jprocont.2010.12.010, https://www.sciencedirect.com/science/article/pii/S095915241000257X. special Issue on Hierarchical and Distributed Model Predictive Control
Nesterov, Y.: A method of solving a convex programming problem with convergence rate \(o(1/k^2)\). Soviet Math. Doklady 27(2), 372–376 (1983)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, Massachusetts (2004)
Patari, N., Venkataramanan, V., Srivastava, A., Molzahn, D.K., Li, N., Annaswamy, A.: Distributed optimization in distribution systems: use cases, limitations, and research needs. IEEE Trans. Power Syst. 1–1 (2021). https://doi.org/10.1109/TPWRS.2021.3132348
Rostampour, V., Haar, O.T., Keviczky, T.: Distributed stochastic reserve scheduling in AC power systems with uncertain generation. IEEE Trans. Power Syst. 34(2), 1005–1020 (2019). https://doi.org/10.1109/TPWRS.2018.2878888
Scaman, K., Bach, F., Bubeck, S., Lee, Y.T., Massoulié, L.: Optimal algorithms for smooth and strongly convex distributed optimization in networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 3027–3036. PMLR, International Convention Centre, Sydney, Australia (2017). http://proceedings.mlr.press/v70/scaman17a.html
Wang, J., Hu, G.: Distributed optimization with coupling constraints in multi-cluster networks based on dual proximal gradient method. arXiv preprint arXiv:2203.00956 (2022)
Wnag, Z., Ong, C.J.: Distributed model predictive control of linear descrete-times systems with local and global cosntraints. Automatica 81, 184–195 (2017)
Yarmoshik, D., Rogozin, A., Khamisov, O., Dvurechensky, P., Gasnikov, A., et al.: Decentralized convex optimization under affine constraints for power systems control. arXiv preprint arXiv:2203.16686 (2022)
Yuan, D., Ho, D.W.C., Jiang, G.P.: An adaptive primal-dual subgradient algorithm for online distributed constrained optimization. IEEE Trans. Cybern. 48, 3045–3055 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rogozin, A., Yarmoshik, D., Kopylova, K., Gasnikov, A. (2022). Decentralized Strongly-Convex Optimization with Affine Constraints: Primal and Dual Approaches. In: Olenev, N., Evtushenko, Y., Jaćimović, M., Khachay, M., Malkova, V., Pospelov, I. (eds) Advances in Optimization and Applications. OPTIMA 2022. Communications in Computer and Information Science, vol 1739. Springer, Cham. https://doi.org/10.1007/978-3-031-22990-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-22990-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-22989-3
Online ISBN: 978-3-031-22990-9
eBook Packages: Computer ScienceComputer Science (R0)