Skip to main content
SpringerLink
Log in

Dual Averaging with Adaptive Random Projection for Solving Evolving Distributed Optimization Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We study a sequential form of the distributed dual averaging algorithm that minimizes the sum of convex functions in a special case where the number of functions increases gradually. This is done by introducing an intermediate ‘pivot’ stage posed as a convex feasibility problem that minimizes average constraint violation with respect to a family of convex sets. Under this approach, we introduce a version of the minimum sum optimization problem that incorporates an evolving design space. Proof of mathematical convergence of the algorithm is complemented by an application problem that involves finding the location of a noisy, mobile source using an evolving wireless sensor network. Results obtained confirm that the new designs in the evolved design space are superior to the ones found in the original design space due to the unique path followed to reach the optimum.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. \(\Vert \cdot \Vert _*\) is the dual norm to \(\Vert \cdot \Vert \). See [6] for details.

  2. With \(\text {int}(S)\) as the interior of the closed convex set S.

  3. Symmetric difference \(\triangle V_t = V_t \triangle V_{t+} = (V_t \cup V_{t+}) \setminus (V_t \cap V_{t+}) \).

References

  1. Bertsekas, D.P.: Incremental gradient, subgradient, and proximal methods for convex optimization: a survey. Optim. Mach. Learn. 2010, 1–38 (2011)

    Google Scholar 

  2. Bertsekas, D.P.: Incremental proximal methods for large scale convex optimization. Math. Program. 129(2), 163–195 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Nedic, A.: Random projection algorithms for convex set intersection problems. In: 2010 49th IEEE Conference on Decision and Control (CDC), pp. 7655–7660. IEEE (2010)

  4. Nedic, A., Ozdaglar, A.: Distributed subgradient methods for multi-agent optimization. IEEE Trans. Autom. Control 54(1), 48–61 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Rabbat, M.G., Nowak, R.D.: Quantized incremental algorithms for distributed optimization. IEEE J. Sel. Areas Commun. 23(4), 798–808 (2005)

    Article  Google Scholar 

  6. Duchi, J.C., Agarwal, A., Wainwright, M.J.: Dual averaging for distributed optimization: convergence analysis and network scaling. IEEE Trans. Autom. Control 57(3), 592–606 (2012)

    Article  MathSciNet  Google Scholar 

  7. Johansson, B., Rabi, M., Johansson, M.: A randomized incremental subgradient method for distributed optimization in networked systems. SIAM J. Optim. 20(3), 1157–1170 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Ram, S.S., Nedić, A., Veeravalli, V.V.: Distributed stochastic subgradient projection algorithms for convex optimization. J. Optim. Theory Appl. 147(3), 516–545 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Censor, Y., De Pierro, A.R., Zaknoon, M.: Steered sequential projections for the inconsistent convex feasibility problem. Nonlinear Anal. Theory Methods Appl. 59(3), 385–405 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  10. De Pierro, A.R., Iusem, A.: A relaxed version of Bregman’s method for convex programming. J. Optim. Theory Appl. 51(3), 421–440 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  11. Censor, Y., Lent, A.: An iterative row-action method for interval convex programming. J. Optim. Theory Appl. 34(3), 321–353 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  12. Censor, Y., Motova, A., Segal, A.: Perturbed projections and subgradient projections for the multiple-sets split feasibility problem. J. Math. Anal. Appl. 327(2), 1244–1256 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  13. Byrne, C.: Bregman-Legendre multidistance projection algorithms for convex feasibility and optimization. Stud. Comput. Math. 8, 87–99 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  14. Aharoni, R., Berman, A., Censor, Y.: An interior points algorithm for the convex feasibility problem. Adv. Appl. Math. 4(4), 479–489 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chinneck, J.W.: Feasibility and Infeasibility in Optimization: Algorithms and Computational Methods, vol. 118. Springer, Berlin (2007)

    MATH  Google Scholar 

  16. Amaldi, E., Pfetsch Jr., M.E., Trotter L.E.: Some structural and algorithmic properties of the maximum feasible subsystem problem. In: Integer Programming and Combinatorial Optimization, pp. 45–59. Springer (1999)

  17. Pfetsch, M.E.: Branch-and-Cut for the Maximum Feasible Subsystem Problem. Konrad-Zuse-Zentrum fr Informationstechnik, Berlin (2005)

    MATH  Google Scholar 

  18. Karger, D.R.: Global min-cuts in rnc, and other ramifications of a simple min-cut algorithm. In: Proceedings of 4th Annual ACM-SIAM Symposium on Discrete Algorithms, vol. 93 (1993)

  19. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38(3), 367–426 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  20. Wang, M., Bertsekas, D.P.: Incremental constraint projection-proximal methods for nonsmooth convex optimization. Technical report, MIT (2013)

  21. Deutsch, F., Hundal, H.: The rate of convergence for the cyclic projections algorithm III: regularity of convex sets. J. Approx. Theory 155(2), 155–184 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  22. Hero, A.O., Cochran, D.: Sensor management: past, present, and future. Sens. J., IEEE 11(12), 3064–3075 (2011)

    Article  Google Scholar 

  23. Cattivelli, F.S., Lopes, C.G., Sayed, A.H.: Diffusion recursive least-squares for distributed estimation over adaptive networks. IEEE Trans. Signal Process. 56(5), 1865–1877 (2008)

    Article  MathSciNet  Google Scholar 

  24. Kekatos, V., Giannakis, G.B.: Distributed robust power system state estimation. IEEE Trans. Power Syst. 28(2), 1617–1626 (2013)

    Article  Google Scholar 

  25. Rabbat, M., Nowak, R.: Distributed optimization in sensor networks. In: Proceedings of the 3rd International Symposium on Information Processing in Sensor Networks, pp. 20–27. ACM (2004)

  26. Huber, P.J., et al.: Robust estimation of a location parameter. Ann. Math. Stat. 35(1), 73–101 (1964)

    Article  MathSciNet  MATH  Google Scholar 

  27. Léger, J.B., Kieffer, M.: Guaranteed robust distributed estimation in a network of sensors. In: 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 3378–3381. IEEE (2010)

  28. Delouille, V., Neelamani, R., Baraniuk, R.: Robust distributed estimation in sensor networks using the embedded polygons algorithm. In: Proceedings of the 3rd International Symposium on Information processing in Sensor Networks, pp. 405–413. ACM (2004)

  29. Moore, D., Leonard, J., Rus, D., Teller, S.: Robust distributed network localization with noisy range measurements. In: Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems, pp. 50–61. ACM (2004)

  30. Li, Q., Wong, W.: Optimal estimator for distributed anonymous observers. J. Optim. Theory Appl. 140(1), 55–75 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  31. Xiao, L., Boyd, S., Lall, S.: A scheme for robust distributed sensor fusion based on average consensus. In: Fourth International Symposium on Information Processing in Sensor Networks, 2005. IPSN 2005, pp. 63–70. IEEE (2005)

Download references

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Shreyas Vathul Subramanian, Daniel A. DeLaurentis & Dengfeng Sun

Authors
  1. Shreyas Vathul Subramanian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel A. DeLaurentis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dengfeng Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shreyas Vathul Subramanian.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Subramanian, S.V., DeLaurentis, D.A. & Sun, D. Dual Averaging with Adaptive Random Projection for Solving Evolving Distributed Optimization Problems. J Optim Theory Appl 170, 493–511 (2016). https://doi.org/10.1007/s10957-016-0932-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-016-0932-z

Keywords

Mathematics Subject Classification

Search

Navigation