Duality of nonconvex optimization with positively homogeneous functions

Abstract

We consider an optimization problem with positively homogeneous functions in its objective and constraint functions. Examples of such positively homogeneous functions include the absolute value function and the p-norm function, where p is a positive real number. The problem, which is not necessarily convex, extends the absolute value optimization proposed in Mangasarian (Comput Optim Appl 36:43–53, 2007). In this work, we propose a dual formulation that, differently from the Lagrangian dual approach, has a closed-form and some interesting properties. In particular, we discuss the relation between the Lagrangian duality and the one proposed here, and give some sufficient conditions under which these dual problems coincide. Finally, we show that some well-known problems, e.g., sum of norms optimization and the group Lasso-type optimization problems, can be reformulated as positively homogeneous optimization problems.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Alizadeh, F., Goldfarb, D.: Second-order cone programming. Math. Program. 95(1), 3–51 (2003)

    MathSciNet  Article  MATH  Google Scholar 

  2. 2.

    Aravkin, A.Y., Burke, J.V., Drusvyatskiy, D., Friedlander, M.P., MacPhee, K.: Foundations of gauge and perspective duality. arXiv preprint arXiv:1702.08649 (2017)

  3. 3.

    Caccetta, L., Qu, B., Zhou, G.: A globally and quadratically convergent method for absolute value equations. Comput. Optim. Appl. 48(1), 45–58 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  4. 4.

    Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14(10), 707–710 (2007)

    Article  Google Scholar 

  5. 5.

    Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, IEEE, pp. 3869–3872 (2008)

  6. 6.

    Eldar, Y.C., Mishali, M.: Robust recovery of signals from a structured union of subspaces. IEEE Trans. Inf. Theory 55(11), 5302–5316 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  7. 7.

    Freund, R.M.: Dual gauge programs, with applications to quadratic programming and the minimum-norm problem. Math. Program. 38(1), 47–67 (1987)

    MathSciNet  Article  MATH  Google Scholar 

  8. 8.

    Friedlander, M.P., Macedo, I., Pong, T.K.: Gauge optimization and duality. SIAM J. Optim. 24(4), 1999–2022 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  9. 9.

    Hu, S., Huang, Z.: A note on absolute value equations. Optim. Lett. 4(3), 417–424 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  10. 10.

    Hu, S., Huang, Z., Zhang, Q.: A generalized Newton method for absolute value equations associated with second order cones. J. Comput. Appl. Math. 235(5), 1490–1501 (2011)

    MathSciNet  Article  MATH  Google Scholar 

  11. 11.

    Kirmaci, U.S., Bakula, M.K., Özdemir, M.E., Pecaric, J.E.: On some inequalities for \(p-\)norms. J. Inequal. Pure Appl. Math. 9(1), 1–8 (2008)

    MathSciNet  MATH  Google Scholar 

  12. 12.

    Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs with Equilibrium Constraints. Cambridge University Press, Cambridge (1996)

    Google Scholar 

  13. 13.

    Mangasarian, O.L.: Absolute value equation solution via concave minimization. Optim. Lett. 1(1), 3–8 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  14. 14.

    Mangasarian, O.L.: Absolute value programming. Comput. Optim. Appl. 36(1), 43–53 (2007)

    MathSciNet  Article  MATH  Google Scholar 

  15. 15.

    Mangasarian, O.L.: A generalized Newton method for absolute value equations. Optim. Lett. 3(1), 101–108 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  16. 16.

    Mangasarian, O.L., Meyer, R.R.: Absolute value equations. Linear Algebra Appl. 419(2), 359–367 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  17. 17.

    Meier, L., van de Geer, S., Bühlmann, P.: The group Lasso for logistic regression. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 70(1), 53–71 (2008)

    MathSciNet  Article  MATH  Google Scholar 

  18. 18.

    Miao, X., Yang, J., Hu, S.: A generalized Newton method for absolute value equations associated with circular cones. Appl. Math. Comput. 269, 155–168 (2015)

    MathSciNet  Google Scholar 

  19. 19.

    Mourad, N., Reilly, J.P.: Minimizing nonconvex functions for sparse vector reconstruction. IEEE Trans. Signal Process. 58(7), 3485–3496 (2010)

    MathSciNet  Article  MATH  Google Scholar 

  20. 20.

    Prokopyev, O.: On equivalent reformulations for absolute value equations. Comput. Optim. Appl. 44(3), 363–372 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  21. 21.

    Rohn, J.: A theorem of the alternatives for the equation \(ax+ b|x|= b\). Linear Multilinear Algebra 52(6), 421–426 (2004)

    MathSciNet  Article  MATH  Google Scholar 

  22. 22.

    Rohn, J.: An algorithm for solving the absolute value equation. Electron. J. Linear Algebra 18(5), 589–599 (2009)

    MathSciNet  MATH  Google Scholar 

  23. 23.

    Stojnic, M., Parvaresh, F., Hassibi, B.: On the reconstruction of block-sparse signals with an optimal number of measurements. IEEE Trans. Signal Process. 57(8), 3075–3085 (2009)

    MathSciNet  Article  MATH  Google Scholar 

  24. 24.

    Wolf, G.W.: Facility Location: Concepts, Models, Algorithms and Case Studies. Taylor & Francis, Routledge (2011)

    Google Scholar 

  25. 25.

    Xue, G., Ye, Y.: An efficient algorithm for minimizing a sum of \(p-\)norms. SIAM J. Optim. 10(2), 551–579 (2000)

    MathSciNet  Article  MATH  Google Scholar 

  26. 26.

    Yamanaka, S., Fukushima, M.: A branch-and-bound method for absolute value programs. Optimization 63(2), 305–319 (2014)

    MathSciNet  Article  MATH  Google Scholar 

  27. 27.

    Yuan, M., Lin, Y.: Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 68(1), 49–67 (2006)

    MathSciNet  Article  MATH  Google Scholar 

  28. 28.

    Zhang, C., Wei, Q.J.: Global and finite convergence of a generalized Newton method for absolute value equations. J. Optim. Theory Appl. 143(2), 391–403 (2009)

    MathSciNet  Article  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Prof. Ellen. H. Fukuda for helpful comments and suggestions. This work was supported in part by a Grant-in-Aid for Scientific Research (C) (17K00032) from Japan Society for the Promotion of Science.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Shota Yamanaka.

Appendix

Appendix

The following proposition shows that the dual of the p-norm function is the \(\infty \)-norm even when p is less than 1.

Proposition A.1

Suppose that \(p \in (0, 1)\). Then, the dual of the p-norm function is equal to the \(\infty \)-norm.

Proof

Let \(y \in \mathbb {R}^n\) be an arbitrary vector. If \(y = 0\), this proposition clearly holds. If \(y \ne 0\), from Definition 2.3, we obtain

$$\begin{aligned} \Vert y \Vert _p^*= & {} \sup \{ x^T y \, | \, \Vert x \Vert _p \le 1 \} \\\le & {} \sup \{ | x^T y | \, | \, \Vert x \Vert _p \le 1 \} \\\le & {} \sup \biggl \{ \sum _{i=1}^n | x_i | | y_i | \, | \, \Vert x \Vert _p \le 1 \biggr \} \\\le & {} \max _j | y_j | \left( \sup \biggl \{ \sum _{i=1}^n | x_i | \, | \, \Vert x \Vert _p \le 1 \biggr \} \right) \\= & {} \max _j | y_j | \biggl ( \sup \{ \Vert x \Vert _1 \, | \, \Vert x \Vert _p \le 1 \} \biggr ). \end{aligned}$$

Since \(p \in (0, 1)\), we note that \(\Vert x \Vert _1 \le \Vert x \Vert _p\) holds [11]. Then, we have

$$\begin{aligned} \Vert y \Vert _p^* \le \max _j | y_j | \biggl ( \sup \{ \Vert x \Vert _p \, | \, \Vert x \Vert _p \le 1 \} \biggr ) = \max _j | y_j | = \Vert y \Vert _\infty . \end{aligned}$$

Now, take an arbitrary \(i_0 \in \mathop {\mathrm{argmax}}\limits _i | y_i |\), and define \(\bar{x}_i\) as follows:

$$\begin{aligned} \bar{x}_i = \left\{ \begin{array}{ll} {\text {sign}}(y_i), &{} \quad \mathrm {if} \quad i = i_0, \\ 0, &{} \quad \mathrm {otherwise}, \end{array} \right. \end{aligned}$$

where

$$\begin{aligned} {\text {sign}}(y_i) = \left\{ \begin{array}{ll} 1, &{} \quad \mathrm {if} \quad y_i > 0, \\ 0, &{} \quad \mathrm {if} \quad y_i = 0, \\ -\,1, &{} \quad \mathrm {if} \quad y_i < 0. \end{array} \right. \end{aligned}$$

Then, \(\Vert \bar{x} \Vert _p = 1\) and we have

$$\begin{aligned} \Vert y \Vert _p^* = \sup \{ x^T y \, | \, \Vert x \Vert _p \le 1 \} \ge \bar{x}^T y = \max _i |y_i | = \Vert y \Vert _\infty , \end{aligned}$$

which completes the proof. \(\square \)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yamanaka, S., Yamashita, N. Duality of nonconvex optimization with positively homogeneous functions. Comput Optim Appl 71, 435–456 (2018). https://doi.org/10.1007/s10589-018-0018-y

Download citation

Keywords

  • Positively homogeneous functions
  • Duality
  • Nonconvex optimization