Skip to main content

Advertisement

Log in

Bounding duality gap for separable problems with linear constraints

Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We consider the problem of minimizing a sum of non-convex functions over a compact domain, subject to linear inequality and equality constraints. Approximate solutions can be found by solving a convexified version of the problem, in which each function in the objective is replaced by its convex envelope. We propose a randomized algorithm to solve the convexified problem which finds an \(\epsilon \)-suboptimal solution to the original problem. With probability one, \(\epsilon \) is bounded by a term proportional to the maximal number of active constraints in the problem. The bound does not depend on the number of variables in the problem or the number of terms in the objective. In contrast to previous related work, our proof is constructive, self-contained, and gives a bound that is tight.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Anderson, E., Lewis, A.: An extension of the simplex algorithm for semi-infinite linear programming. Math. Program. 44(1–3), 247–269 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  2. Aubin, J., Ekeland, I.: Estimates of the duality gap in nonconvex optimization. Math. Oper. Res. 1(3), 225–245 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  3. Barvinok, A.: A Course in Convexity, vol. 54. American Mathematical Society, Providence (2002)

    MATH  Google Scholar 

  4. Barvinok, A.I.: Problems of distance geometry and convex properties of quadratic maps. Discret. Comput. Geom. 13(1), 189–202 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  5. Benoist, J., Hiriart-Urruty, J.B.: What is the subdifferential of the closed convex hull of a function? SIAM J. Math. Anal. 27(6), 1661–1679 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bertsekas, D.: Nonlinear Programming. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  7. Bertsekas, D.: Convex Optimization Theory. Athena Scientific, Belmont (2009)

    MATH  Google Scholar 

  8. Bertsekas, D., Lauer, G., Sandell, N., Posbergh, T.: Optimal short-term scheduling of large-scale power systems. IEEE Trans. Autom. Control 28(1), 1–11 (1983)

    Article  MATH  Google Scholar 

  9. Bertsekas, D.P.: Constrained Optimization and Lagrange Multipler Methods. Athena Scientific, Belmont (1982)

    MATH  Google Scholar 

  10. Bolte, J., Daniilidis, A., Lewis, A.S.: Generic optimality conditions for semialgebraic convex programs. Math. Oper. Res. 36(1), 55–70 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Borwein, J.M., Vanderwerff, J.D.: Convex Functions: Constructions, Characterizations and Counterexamples. Cambridge University Press, Cambridge (2010)

    Book  MATH  Google Scholar 

  12. Bouaziz, S., Tagliasacchi, A., Pauly, M.: Sparse iterative closest point. In: Computer Graphics Forum, vol. 32, pp. 113–123. Wiley Online Library (2013)

  13. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  14. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  15. Chartrand, R.: Nonconvex splitting for regularized low-rank \(+\) sparse decomposition. IEEE Trans. Signal Process. 60(11), 5810–5819 (2012)

    Article  MathSciNet  Google Scholar 

  16. Chartrand, R., Wohlberg, B.: A nonconvex admm algorithm for group sparsity with sparse groups. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6009–6013. IEEE (2013)

  17. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  18. Derbinsky, N., Bento, J., Elser, V., Yedidia, J.S.: An improved three-weight message-passing algorithm. arXiv preprint arXiv:1305.1961 (2013)

  19. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  20. Eckstein, J., Fukushima, M.: Some reformulations and applications of the alternating direction method of multipliers. Large Scale Optimization: State of the Art, pp. 119–138. Springer, New York (1993)

    Google Scholar 

  21. Fazel, M., Chiang, M.: Network utility maximization with nonconcave utilities using sum-of-squares method. In: Proceedings of the European Control Conference, pp. 1867–1874 (2005). doi:10.1109/CDC.2005.1582432

  22. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)

    MATH  Google Scholar 

  23. Fortin, M., Glowinski, R.: On decomposition-coordination methods using an augmented Lagrangian. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)

    Google Scholar 

  24. Fukushima, M.: Application of the alternating direction method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1, 93–111 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)

    Google Scholar 

  26. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  27. Gilimyanov, R., Zhuang, H.: Power allocation in OFDMA networks: An ADMM approach. In: 5th Traditional Youth Summer School on Control, Information, and Optimization (2013)

  28. Glowinski, R.: On alternating direction methods of multipliers: a historical perspective. In: W. Fitzgibbon, Y.A. Kuznetsov, P. Neittaanmki, O. Pironneau (eds.) Modeling, Simulation and Optimization for Science and Technology, Computational Methods in Applied Sciences, vol. 34, pp. 59–82. Springer (2014). doi:10.1007/978-94-017-9054-3_4

  29. Glowinski, R., Marrocco, A.: Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualité, d’une classe de problems de Dirichlet non lineares. Revue Française d’Automatique, Informatique, et Recherche Opérationelle 9, 41–76 (1975)

    MathSciNet  MATH  Google Scholar 

  30. Glowinski, R., Tallec, P.L.: Augmented Lagrangian methods for the solution of variational problems. Tech. Rep. 2965, University of Wisconsin-Madison (1987)

  31. Goemans, M.X., Williamson, D.P.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM 42, 1115–1145 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  32. He, B., Yuan, X.: On the \({O}(1/n)\) convergence rate of the Douglas-Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I: Part 1: Fundamentals. Springer, Berlin (1996)

    MATH  Google Scholar 

  34. Hong, M., Luo, Z.Q.: On the linear convergence of the alternating direction method of multipliers. arXiv preprint arXiv:1208.3922 (2012)

  35. Horst, R., Pardalos, P.M., Thoai, N.V.: Introduction to Optimization. Kluwer Academic Publishers, Dordrecht (2000)

    MATH  Google Scholar 

  36. Kanamori, T., Takeda, A.: Non-convex optimization on Stiefel manifold and applications to machine learning. In: T. Huang, Z. Zeng, C. Li, C. Leung (eds.) Neural Information Processing, Lecture Notes in Computer Science, vol. 7663, pp. 109–116. Springer (2012). doi:10.1007/978-3-642-34475-6_14

  37. Lemaréchal, C., Renaud, A.: A geometric study of duality gaps, with applications. Math. Program. 90(3), 399–427 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  38. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  39. Magnusson, S., Weeraddana, P.C., Fischione, C.: A distributed approach for the optimal power flow problem based on ADMM and sequential convex approximations. arXiv preprint arXiv:1401.4621 (2014)

  40. Motwani, R.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)

    Book  MATH  Google Scholar 

  41. Oberman, A.M.: The convex envelope is the solution of a nonlinear obstacle problem. Proc. Am. Math. Soc. 135(6), 1689–1694 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  42. Oberman, A.M.: Computing the convex envelope using a nonlinear partial differential equation. Math. Models Methods Appl. Sci. 18(05), 759–780 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  43. Pataki, G.: Cone-LP’s and Semidefinite Programs: Geometry and a Simplex-Type Method. Integer Programming and Combinatorial Optimization, pp. 162–174. Springer, Berlin (1996)

    Book  Google Scholar 

  44. Pataki, G.: On the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues. Math. Oper. Res. 23(2), 339–358 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  45. Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3, 28–41 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  46. Rikun, A.D.: A convex envelope formula for multilinear functions. J. Glob. Optim. 10(4), 425–437 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  47. Rockafellar, R.: Convex Analysis, vol. 28. Princeton University Press, Princeton (1970)

    Book  MATH  Google Scholar 

  48. Skaf, J., Boyd, S.: Techniques for exploring the suboptimal set. Optim. Eng. 11(2), 319–337 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  49. Starr, R.M.: Quasi-equilibria in markets with nonconvex preferences. Econometrica 37, 25–38 (1969)

    Article  MATH  Google Scholar 

  50. Tawarmalani, M., Sahinidis, N.: Convexification and Global Optimization in Continuous and Mixed-integer Nonlinear Programming: Theory, Algorithms, Software, and Applications, vol. 65. Springer, New York (2002)

  51. Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 29(1), 119–138 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  52. Udell, M., Boyd, S.: Maximizing a sum of sigmoids. Available at http://www.stanford.edu/boyd/papers/max_sum_sigmoids.html (2013)

  53. Valadier, M.: Intégration de convexes fermés notamment d’épigraphes inf-convolution continue. ESAIM 4(R2), 57–73 (1970)

    MathSciNet  MATH  Google Scholar 

  54. Vujanic, R., Esfahani, P.M., Goulart, P., Mariethoz, S., Morari, M.: Vanishing duality gap in large scale mixed-integer optimization: a solution method with power system applications. J. Math. Program. (2014)

  55. Zhang, Y.: Recent advances in alternating direction methods: practice and theory. In: IPAM Workshop on Continuous Optimization (2010)

Download references

Acknowledgments

The authors thank Haitham Hindi, Ernest Ryu and the anonymous reviewers for their very careful readings of and comments on early drafts of this paper, and Jon Borwein and Julian Revalski for their generous advice on the technical lemmas in the appendix. This work was developed with support from the National Science Foundation Graduate Research Fellowship program (under Grant No. DGE-1147470), the Gabilan Stanford Graduate Fellowship, the Gerald J. Lieberman Fellowship, and the DARPA X-DATA Program

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Madeleine Udell.

Appendices

Appendix 1: The dual of the dual is the convexified problem

In this appendix, we prove that the dual of the dual of \(\mathcal {P}\) is the convexified problem \(\hat{\mathcal {P}}\).

Before we begin, note that the convex envelope has a close connection to duality. Let \(f^*(y) = \sup (y^T x - f(x)) = - \inf (f(x) - y^T x)\) be the (Fenchel) conjugate of f. Then \(\hat{f}(x) = f^{**}(x)\) is the biconjugate of f [47]. The conjugate function arises naturally when taking the dual of a problem, as we show below. Hence it should come as no suprise that the biconjugate appears upon taking the dual twice.

Below, we refer to the dual of the dual problem as the dual dual problem, the dual function of the dual problem as the dual dual function, and the variables in the dual dual problem as the dual dual variables.

Recall the primal problem, which we write as

$$\begin{aligned} \begin{array}{ll} \hbox {minimize} &{} f(x) = \sum \limits _{i=1}^n f_i(x_i) \\ \hbox {subject to} &{} A x \le b \\ &{} G x = h. \end{array} \end{aligned}$$

We can write the Lagrangian of the primal problem as

$$\begin{aligned} L (x,\lambda ,\mu ) = \sum _{i=1}^n f_i(x_i) + \lambda ^T (A x - b) + \mu ^T(G x - h), \end{aligned}$$

with dual variables \(\lambda \ge 0\) and \(\mu \). The dual function \(g(\lambda , \mu )\) is the minimum of the Lagrangian over x,

$$\begin{aligned} g(\lambda , \mu )= & {} \inf _x L (x,\lambda ,\mu ) \\= & {} \inf _x \sum _{i=1}^n f_i(x_i) + \lambda ^T (A x - b) + \mu ^T(G x - h) \\= & {} \sum _{i=1}^n \inf _{x_i} (f_i(x_i) - \gamma _i x_i) - \lambda ^T b - \mu ^T h \\= & {} \sum _{i=1}^n -f_i^*(\gamma _i) - \lambda ^T b - \mu ^T h, \\ \end{aligned}$$

where we have defined \(\gamma = - A^T \lambda - G^T \mu \) in the second to last equality and used the relation \(f^*(y) = - \inf (f(x) - y^T x)\) in the last.

The dual problem is to maximize the dual function over \(\mu \) and \(\lambda \) with \(\lambda \ge 0\):

$$\begin{aligned} \begin{array}{ll} \hbox {maximize} &{} \sum \limits _{i=1}^n -f_i^*(\gamma _i) - \lambda ^T b - \mu ^T h\\ \hbox {subject to} &{} \gamma = - A^T \lambda - G^T \mu \\ &{} \lambda \ge 0. \end{array} \end{aligned}$$

The conjugate function \(f_i^*\) is a pointwise supremum of affine functions, and so is always convex even if \(f_i\) is not. Hence the dual problem is a concave maximization problem.

To take the dual of the dual, we perform exactly the same computations again on the dual problem now instead of the primal. The dual Lagrangian is

$$\begin{aligned} L_D (\lambda ,\mu ,\gamma ,x,y) = \sum _{i=1}^n -f_i^*(\gamma _i) - \lambda ^T b - \mu ^T h + x^T(\gamma + A^T \lambda + G^T) + s^T \lambda , \end{aligned}$$

with dual dual variables \(s \ge 0\) and x. We maximize the dual Lagrangian over the dual variables \(\lambda \), \(\mu \), and \(\gamma \) to form the dual dual function

$$\begin{aligned} g_D(x, s)= & {} \sup _{\lambda \ge 0, \mu , \gamma } L_D (\lambda ,\mu ,\gamma ,x,y) \\= & {} \sup _{\lambda \ge 0,\mu ,\gamma } \sum _{i=1}^n -f_i^*(\gamma _i) - \lambda ^T b - \mu ^T h + x^T(\gamma + A^T \lambda + G^T) + s^T \lambda \\= & {} \sup _{\lambda \ge 0,\mu } \sum _{i=1}^n f_i^{**}(x_i) + \lambda ^T (Ax + s - b) + \mu ^T (Gx - h), \end{aligned}$$

using now the relation \(f^*(y) = \sup (y^T x - f(x))\). This is finite only if \(Ax + s - b \le 0\) and \(Gx - h = 0\). So we see

$$\begin{aligned} g_D(x, s) = \sum _{i=1}^n f_i^{**}(x_i) \end{aligned}$$

so long as these equalities are satisfied.

To form the dual dual problem, we minimize the dual dual function over x and \(s \ge 0\):

$$\begin{aligned} \begin{array}{ll} \hbox {minimize} &{} \sum \limits _{i=1}^n f_i^{**}(x_i) \\ \hbox {subject to} &{} A x \le b \\ &{} G x = h, \end{array} \end{aligned}$$

where we have solved for \(s = b - Ax\). Hence we see that we have recovered the convexified problem by dualizing the primal twice.

Appendix 2: Well-posedness

The following theorem characterizes the set of vectors in the dual space for which linear optimization over a compact set S is well-posed.

Theorem 3

(Well-posedness of linear optimization) Suppose S is a compact set in \({\mathbf{R}}^n\). Then the set of \(w\in {\mathbf{R}}^n\) for which the maximizer of \(w^Tx\) over S is not unique has (Lebesgue) measure zero.

This result is well-known; for example, it follows from [10, Sect. 2], taking into account that if \(S \subseteq {\mathbf{R}}^n\) is compact, then so is its convex hull \(K = \mathbf{conv}(S)\) and the set of extreme points of S and K coincide. In fact, one can derive much stronger results using, for example, Alexandrov’s theorem for convex functions to show quadratic decay, or finite identifiability in the case of semialgebraic functions. However, our purpose here is more modest; we merely prove the weaker result stated as Theorem 3 so that this paper may be self-contained.

Before proceeding to a proof, however, let us make sense of the statement of the theorem. By definition, the maximizer of a linear functional over a set S is a face R of S. The maximizer is unique if and only if R is a zero-dimensional face (i.e., an extreme point). Only an outward normal to a face will be maximized on that face.

It is easy to see that the theorem is true for polyhedral sets S. For each face of the polyhedron that is not extreme, the set of vectors maximized by that face (the set of outward normals to the face, i.e., the normal cone) will have dimension smaller than n. A polyhedron has only a bounded number of faces, so the union of these sets still has measure zero.

On the opposite extreme, consider the unit sphere. A sphere has an infinite number of faces. But every face is extreme, and every vector w has a unique maximizer.

The difficulty comes when we consider cylindrical sets: those constructed as the Cartesian product of a sphere and a cube. Here, every outward normal to the “sides” of the cylinder is a vector whose maximum over the set is not extreme. That is, we find an uncountably infinite number of faces (parametrized by the boundary of the sphere) that are not extreme points.

Proof

Let \(I_S: {\mathbf{R}}^n \rightarrow {\mathbf{R}}\) be the indicator function of S. S is compact, so the convex conjugate \(I^*_S(y) = \sup _x y^Tx - I_S(x)\) of \(I_S\) is finite for every \(y \in {\mathbf{R}}^n\). Rachemacher’s Theorem [11, Theorem 2.5.1] states that a convex function \(g:{\mathbf{R}}^n \rightarrow {\mathbf{R}}\) is differentiable almost everywhere with respect to Lebesgue measure on \({\mathbf{R}}^n\). Furthermore, if \(I^*_S\) is differentiable at y with \(\nabla I^*_S(y) = x\), then \(y^Tx - I_S(x)\) attains a strong maximum at x [11, Theorem 5.2.3]; that is, there is a unique maximizer of \(y^Tx\) over S.\(\square \)

Clearly, the statement also holds for the minimizers, rather than maximizers, of \(w^Tx\).

The following corollary will be used in the proof of the main theorem of this paper.

Corollary 1

Suppose S is a compact set in \({\mathbf{R}}^n\), and w is a uniform random variable on the unit sphere in \({\mathbf{R}}^n\). Then with probability one, there is a unique minimizer of \(w^Tx\) over S.

Proof

The property of having a unique minimizer exhibits a symmetry along radial lines: there is a unique minimizer of \(w^Tx\) over S if and only if there is a unique minimizer of \((w/\Vert w\Vert _2)^Tx\) over S. A uniform random vector on the unit sphere may be generated by taking a uniform random vector on the unit ball, and normalizing it to lie on the unit sphere. Since the set of directions whose maximizers are not unique has Lebesgue measure zero, the vectors on the unit sphere generated in this manner have maximizers that are unique with probability one.

We give one last corollary, which may be of mathematical interest, but is not used elsewhere in this paper.

Corollary 2

Suppose S is a compact set in \({\mathbf{R}}^n\). The union of the normal cones N(x) of all points \(x \in S\) that are not extreme has measure zero.

Proof

A point x minimizes \(y^Tx\) over S if and only if \(y \in N(x)\). A point x is the only minimizer of \(y^Tx\) over S if and only if x is exposed, and hence extreme. Hence no y with a unique minimizer over S lies in the normal cone of a point that is not extreme. Thus the union of the normal cones N(x) of all points \(x \in S\) that are not extreme is a subset of the vectors which do not have a unique maximizer over S, and hence has measure zero.

Appendix 3: Closure

The following lemma technical lemma will be useful in the main body of the paper.

Lemma 4

Let \(S \subset {\mathbf{R}}^n\) be a nonempty compact set, and let \(f: S \rightarrow {\mathbf{R}}\) be lower semi-continuous on S. Then \(\mathbf{conv}(\mathbf{epi}f)\) is closed.

This result follows from [5, Thm. 4.6], since every function defined on a compact set is in particular 1-coercive. The earliest proof known to the authors can be found in [53, p. 69]; for a simpler exposition, see [33, Ch. X, Sect. 1.5]. Here, we provide a self-contained elementary proof for the curious reader.

Proof

Every point \((x,t) \in \mathbf{cl}(\mathbf{conv}(\mathbf{epi}f))\) is a limit of points \((x^k, t^k)\) in \(\mathbf{conv}(\mathbf{epi}f)\). These points can be written as

$$\begin{aligned} (x^k, t^k) = \sum _{i=1}^{n+2} \lambda ^k_i (a^k_i,b^k_i) \end{aligned}$$

with \(\sum _{i=1}^{n+2} \lambda ^k_i = 1\), \(0 \le \lambda ^k_i \le 1\), and \((a^k_i,b^k_i) \in \mathbf{epi}(f)\). Since [0, 1] and S are compact, we can find a subsequence along which each sequence \(a^k_i\) converges to a limit \(a_i \in S\), and each sequence \(\lambda ^k_i\) converges to a limit \(\lambda _i \in [0,1]\).

Let \(P = \{i : \lambda _i > 0\)}. Note that P is not empty, since \(\sum _{i=1}^{n+2} \lambda ^k_i = 1\) for every k. If \(l \in P\), then because the limit t exists, \(\limsup _k b_i^k\) is bounded above. Recall that a lower semi-continuous function is bounded below on a compact domain, so \(b_i^k\) is also bounded below. This shows that for \(i \in P\), every subsequence of \(b_i^k\) has a subsequence that converges to a limit \(b_i\). In particular, we can pick a subsequence \(k_j\) such that simultaneously, for \(i=1,\ldots ,n+2\), \(a_i^{k_j}\), \(b_i^{k_j}\), and \(\lambda _i^{k_j}\) converge along the subsequence \(k_j\) to \(a_i\), \(b_i\), and \(\lambda _i\), respectively.

Define \(S_P = \sum _{i\in P} \lambda _i b_i\). Then along the subsequence \(k_j\), \(\lim _{j \rightarrow \infty } \sum _{i\notin P} \lambda _i^{k_j} b_i^{k_j} = t - S_P\) also exists. Since f is bounded below, \(b_i^k\) are all bounded below, and for \(i \notin P\), \(\lambda _i^k \rightarrow 0\), so \(t- S_P \ge 0\). Therefore (xt) can be written as \(\sum _{i \in P} \lambda _i (a_i,b_i) + (0, t- S_P)\).

Recall that a function is lower semi-continuous if and only if its epigraph is closed. Hence \((a_i, b_i) \in \mathbf{epi}f\) for \(i \in P\). Without loss of generality, suppose \(1 \in P\), and note that \((a_1, b_1 + t - S_P) \in \mathbf{epi}f\), since \(t-S_P\) is non-negative.

Armed with these facts, we see we can write (xt) as a convex combination of points in \(\mathbf{epi}f\),

$$\begin{aligned} (x,t) = \lambda _1 (a_1, b_1 + t-S_P) + \sum _{i\in S, i \ne 1} \lambda _i (a_i,b_i). \end{aligned}$$

Thus every \((x,t) \in \mathbf{cl}(\mathbf{conv}(\mathbf{epi}f))\) can be written as a convex combination of points in \(\mathbf{epi}f\), so \(\mathbf{conv}(\mathbf{epi}f)\) is closed.\(\square \)

Corollary 3

Let \(S \subset {\mathbf{R}}^n\) be a compact set, and let \(f: S \rightarrow {\mathbf{R}}\) be lower semi-continuous on S. Then \(\mathbf{epi}(\hat{f}) = \mathbf{cl}( \mathbf{conv}(\mathbf{epi}f)) = \mathbf{conv}(\mathbf{epi}f)\).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Udell, M., Boyd, S. Bounding duality gap for separable problems with linear constraints. Comput Optim Appl 64, 355–378 (2016). https://doi.org/10.1007/s10589-015-9819-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9819-4

Keywords

Navigation