Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems

Article

Abstract

A new algorithm for solving large-scale convex optimization problems with a separable objective function is proposed. The basic idea is to combine three techniques: Lagrangian dual decomposition, excessive gap and smoothing. The main advantage of this algorithm is that it automatically and simultaneously updates the smoothness parameters which significantly improves its performance. The convergence of the algorithm is proved under weak conditions imposed on the original problem. The rate of convergence is \(O(\frac {1}{k})\), where k is the iteration counter. In the second part of the paper, the proposed algorithm is coupled with a dual scheme to construct a switching variant in a dual decomposition framework. We discuss implementation issues and make a theoretical comparison. Numerical examples confirm the theoretical results.

Keywords

Excessive gap Smoothing technique Lagrangian decomposition Proximal mappings Large-scale problem Separable convex optimization Distributed optimization 

References

  1. 1.
    Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Prentice Hall, New York (1989) MATHGoogle Scholar
  2. 2.
    Boyd, S., Parikh, N., Chu, E., Peleato, B.: Distributed optimization and statistics via alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011) CrossRefGoogle Scholar
  3. 3.
    Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994) MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Connejo, A.J., Mínguez, R., Castillo, E., García-Bertrand, R.: Decomposition Techniques in Mathematical Programming: Engineering and Science Applications. Springer, Berlin (2006) Google Scholar
  5. 5.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002) MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Duchi, J.C., Agarwal, A., Wainwright, M.J.: Dual averaging for distributed optimization: Convergence analysis and network scaling. IEEE Trans. Autom. Control 57(3), 592–606 (2012) MathSciNetCrossRefGoogle Scholar
  7. 7.
    Eckstein, J., Bertsekas, D.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992) MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vols. 1–2. Springer, Berlin (2003) Google Scholar
  9. 9.
    Goldfarb, D., Ma, S.: Fast multiple splitting algorithms for convex optimization. SIAM J. Optim. 22(2), 533–556 (2012) MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Hamdi, A.: Two-level primal-dual proximal decomposition technique to solve large-scale optimization problems. Appl. Math. Comput. 160, 921–938 (2005) MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Han, S.P., Lou, G.: A parallel algorithm for a class of convex programs. SIAM J. Control Optim. 26, 345–355 (1988) MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Hariharan, L., Pucci, F.D.: Decentralized resource allocation in dynamic networks of agents. SIAM J. Optim. 19(2), 911–940 (2008) MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    He, B.S., Tao, M., Xu, M.H., Yuan, X.M.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimization (2011). doi:10.1080/02331934.2011.611885 Google Scholar
  14. 14.
    He, B.S., Yang, H., Wang, S.L.: Alternating directions method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 106, 349–368 (2000) MathSciNetCrossRefGoogle Scholar
  15. 15.
    He, B.S., Yuan, X.M.: On the O(1/n) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012) MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Holmberg, K.: Experiments with primal-dual decomposition and subgradient methods for the uncapacitated facility location problem. Optimization 49(5–6), 495–516 (2001) MathSciNetMATHCrossRefGoogle Scholar
  17. 17.
    Holmberg, K., Kiwiel, K.C.: Mean value cross decomposition for nonlinear convex problem. Optim. Methods Softw. 21(3), 401–417 (2006) MathSciNetMATHCrossRefGoogle Scholar
  18. 18.
    Kojima, M., Megiddo, N., Mizuno, S., et al.: Horizontal and vertical decomposition in interior point methods for linear programs. Technical report, Information Sciences, Tokyo Institute of Technology, Tokyo (1993) Google Scholar
  19. 19.
    Lenoir, A., Mahey, P.: Accelerating convergence of a separable augmented Lagrangian algorithm. Technical report, LIMOS/RR-07-14, pp. 1–34 (2007) Google Scholar
  20. 20.
    Love, R.F., Kraemer, S.A.: A dual decomposition method for minimizing transportation costs in multifacility location problems. Transp. Sci. 7, 297–316 (1973) MathSciNetCrossRefGoogle Scholar
  21. 21.
    Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992) MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Necoara, I., Suykens, J.A.K.: Applications of a smoothing technique to decomposition in convex optimization. IEEE Trans. Autom. Control 53(11), 2674–2679 (2008) MathSciNetCrossRefGoogle Scholar
  23. 23.
    Nedíc, A., Ozdaglar, A.: Distributed subgradient methods for multi-agent optimization. IEEE Trans. Autom. Control 54, 48–61 (2009) CrossRefGoogle Scholar
  24. 24.
    Nesterov, Y.: A method for unconstrained convex minimization problem with the rate of convergence o(1/k 2). Dokl. Akad. Nauk SSSR 269, 543–547 (1983) (Translated as Soviet Math. Dokl.) MathSciNetGoogle Scholar
  25. 25.
    Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Applied Optimization, vol. 87. Kluwer Academic, Dordrecht (2004) MATHGoogle Scholar
  26. 26.
    Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005) MathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005) MathSciNetMATHCrossRefGoogle Scholar
  28. 28.
    Neveen, G., Jochen, K.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007) MathSciNetCrossRefGoogle Scholar
  29. 29.
    Ruszczyński, A.: On convergence of an augmented Lagrangian decomposition method for sparse convex optimization. Math. Oper. Res. 20, 634–656 (1995) MathSciNetMATHCrossRefGoogle Scholar
  30. 30.
    Samar, S., Boyd, S., Gorinevsky, D.: Distributed estimation via dual decomposition. In: Proceedings European Control Conference (ECC), Kos, Greece, pp. 1511–1516 (2007) Google Scholar
  31. 31.
    Spingarn, J.E.: Applications of the method of partial inverses to convex programming: decomposition. Math. Program. Ser. A 32, 199–223 (1985) MathSciNetMATHCrossRefGoogle Scholar
  32. 32.
    Tran-Dinh, Q., Necoara, I., Savorgnan, C., Diehl, M.: An inexact perturbed path-following method for Lagrangian decomposition in large-scale separable convex optimization. Int. Report 12-181, ESATSISTA, KU Leuven, Belgium (2012). SIAM J. Optim., accepted Google Scholar
  33. 33.
    Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7(4), 951–965 (1997) MathSciNetMATHCrossRefGoogle Scholar
  34. 34.
    Tsiaflakis, P., Diehl, M., Moonen, M.: Distributed spectrum management algorithms for multi-user DSL networks. IEEE Trans. Signal Process. 56(10), 4825–4843 (2008) MathSciNetCrossRefGoogle Scholar
  35. 35.
    Tsiaflakis, P., Necoara, I., Suykens, J.A.K., Moonen, M.: Improved dual decomposition based optimization for DSL dynamic spectrum management. IEEE Trans. Signal Process. 58(4), 2230–2245 (2010) MathSciNetCrossRefGoogle Scholar
  36. 36.
    Vania, D.S.E.: Finding approximate solutions for large scale linear programs. Ph.D. Thesis, No. 18188, ETH, Zurich (2009) Google Scholar
  37. 37.
    Venkat, A.N.: Distributed model predictive control: theory and applications. Ph.D. Thesis, University of Wisconsin-Madison (2006) Google Scholar
  38. 38.
    Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006) MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Zhao, G.: A Lagrangian dual method with self-concordant barriers for multistage stochastic convex programming. Math. Program. 102, 1–24 (2005) MathSciNetMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Quoc Tran Dinh
    • 1
    • 2
  • Carlo Savorgnan
    • 1
  • Moritz Diehl
    • 1
  1. 1.Department of Electrical Engineering (ESAT-SCD) and Optimization in Engineering Center (OPTEC)KU LeuvenHeverlee-LeuvenBelgium
  2. 2.Vietnam National UniversityHanoiVietnam

Personalised recommendations