# On the pervasiveness of difference-convexity in optimization and statistics

- 242 Downloads
- 2 Citations

## Abstract

With the increasing interest in applying the methodology of difference-of-convex (dc) optimization to diverse problems in engineering and statistics, this paper establishes the dc property of many functions in various areas of applications not previously known to be of this class. Motivated by a quadratic programming based recourse function in two-stage stochastic programming, we show that the (optimal) value function of a copositive (thus not necessarily convex) quadratic program is dc on the domain of finiteness of the program when the matrix in the objective function’s quadratic term and the constraint matrix are fixed. The proof of this result is based on a dc decomposition of a piecewise \(\hbox {LC}^1\) function (i.e., functions with Lipschitz gradients). Armed with these new results and known properties of dc functions existed in the literature, we show that many composite statistical functions in risk analysis, including the value-at-risk (VaR), conditional value-at-risk (CVaR), optimized certainty equivalent, and the expectation-based, VaR-based, and CVaR-based random deviation functionals are all dc. Adding the known class of dc surrogate sparsity functions that are employed as approximations of the \(\ell _0\) function in statistical learning, our work significantly expands the classes of dc functions and positions them for fruitful applications.

## Mathematics Subject Classification

90C26## Notes

### Acknowledgements

The second author gratefully acknowledges the discussion with Professors Le Thi Hoi An and Pham Dinh Tao in the early stage of this work during his visit to the Université de Lorraine, Metz in June 2016. The three authors acknowledge their fruitful discussion with Professor Defeng Sun at the National University of Singapore during his visit to the University of Southern California. They are also grateful to Professor Marc Teboulle for drawing their attention to the references [5, 6, 7] that introduce and revisit the OCE. The constructive comments of two referees are also gratefully acknowledged. In particular, the authors are particularly grateful to a referee who has been very patient with our repeated misunderstanding of the work [57] that is now correctly summarized at the end of Sect. 3.1.

## References

- 1.Ahn, M., Pang, J.S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Optim.
**27**, 1637–1665 (2017)MathSciNetCrossRefzbMATHGoogle Scholar - 2.Alexandroff, A.D.: Surfaces represented by the difference of convex functions. Doklady Akademii Nauk SSSR (N.S.)
**72**, 613–616 (1950). [English translation: Siberian Èlektron. Mathetical. Izv.**9**, 360–376 (2012)]Google Scholar - 3.Alvarado, A., Scutari, G., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process.
**62**, 2984–2998 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 4.Bačák, M., Borwein, J.M.: On difference convexity of locally Lipschitz functions. Optimization
**60**, 961–978 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 5.Ben-Tal, A., Teboulle, M.: Expected utility, penalty functions and duality in stochastic nonlinear programmi. Manage. Sci.
**32**, 14451466 (1986)CrossRefzbMATHGoogle Scholar - 6.Ben-Tal, A., Teboulle, M.: Penalty functions and duality in stochastic programming via \(\varphi \)-divergence functionals. Math. Oper. Res.
**12**, 224–240 (1987)MathSciNetCrossRefzbMATHGoogle Scholar - 7.Ben-Tal, A., Teboulle, M.: An old-new concept of convex risk measures: the optimized certainty equivalent. Math. Finance
**17**, 449–476 (2007)MathSciNetCrossRefzbMATHGoogle Scholar - 8.Birge, J.R., Louveaux, F.: Introduction to Stochastic Programming. Springer Series in Operations Research. Springer, New York (1997)zbMATHGoogle Scholar
- 9.Chang, T.-H., Hong, M., Pang, J.S.: Local minimizers and second-order conditions in composite piecewise programming via directional derivatives. Preprint arXiv:1709.05758 (2017)
- 10.Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)zbMATHGoogle Scholar
- 11.Cui, Y., Pang, J.-S., Sen, B.: Composite difference-max programs for modern statistical estimation problems. Preprint arXiv:1803.00205 (2018)
- 12.Eaves, B.C.: On quadratic programming. Manage. Sci.
**17**, 698–711 (1971)CrossRefzbMATHGoogle Scholar - 13.Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, Volumes I and II. Springer, New York (2003)zbMATHGoogle Scholar
- 14.Fan, J., Xue, L., Zou, H.: Strong oracle optimality of folded concave penalized estimation. Ann. Stat.
**42**(2014), 819–849 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 15.Giannessi, F., Tomasin, E.: Nonconvex quadratic programs, linear complementarity problems, and integer linear programs. In: Conti, R., Ruberti, A. (eds) Fifth Conference on Optimization Techniques (Rome 1973), Part I, Lecture Notes in Computer Science, Vol. 3, pp. 437–449. Springer, Berlin (1973)Google Scholar
- 16.Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Math. Program. Ser. B. (2017). https://doi.org/10.1007/s10107-017-1181-0
- 17.Hahn, G., Banergjee, M., Sen, B.: Parameter Estimation and Inference in a Continuous Piecewise Linear Regression Model. Manuscript, Department of Statistics. Columbia University, New York (2016)Google Scholar
- 18.Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math.
**9**, 707–713 (1959)MathSciNetCrossRefzbMATHGoogle Scholar - 19.Hiriart-Urruty, J.B.: Generalized differentiability, duality and optimization for problems dealing with differences of convex functions. In: Ponstein, J. (eds) Convexity and Duality in Optimization. Proceedings of the Symposium on Convexity and Duality in Optimization Held at the University of Groningen, The Netherlands June 22, 1984, pp. 37–70 (1985)Google Scholar
- 20.Horst, R., Thoai, N.V.: D.C. programming: overview. J. Optim. Theory Appl.
**103**, 1–43 (1999)MathSciNetCrossRefzbMATHGoogle Scholar - 21.Hu, J., Mitchell, J.E., Pang, J.S.: An LPCC approach to nonconvex quadratic programs. Math. Program.
**133**, 243–277 (2012)MathSciNetCrossRefzbMATHGoogle Scholar - 22.Jara-Moroni, F., Pang, J.-S., Wächter, A.: A study of the difference-of-convex approach for solving linear programs with complementarity constraints. Math. Program.
**169**(1), 221–254 (2018)MathSciNetCrossRefzbMATHGoogle Scholar - 23.Le Thi, H.A., Pham, D.T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res.
**133**, 25–46 (2005)MathSciNetzbMATHGoogle Scholar - 24.Le Thi, H.A., Pham, D.T.: Recent advances in DC programming and DCA. Trans. Comput. Collect. Intell.
**8342**, 1–37 (2014)Google Scholar - 25.Le Thi, H.A., Pham, D.T.: DC Programming and DCA: Thirty years of Developments Manuscript. University of Lorraine, Lorraine (2016)zbMATHGoogle Scholar
- 26.Le Thi, H.A., Pham, D.T., Vo, X.T.: DC approximation approaches for sparse optimization. Eur. J. Oper. Res.
**244**, 26–46 (2015)MathSciNetCrossRefzbMATHGoogle Scholar - 27.Lee, G.M., Tam, N.N., Yen, N.D.: Quadratic Programming and Affine Variational Inequalities A Qualitative Study. Springer, New York (2005)zbMATHGoogle Scholar
- 28.Luo, Z.Q., Tseng, P.: Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim.
**2**, 43–54 (1992)MathSciNetCrossRefzbMATHGoogle Scholar - 29.Luo, Z.Q., Pang, J.S., Ralph, D.: Mathematical Programs With Equilibrium Constraints. Cambridge University Press, Cambridge (1996)CrossRefzbMATHGoogle Scholar
- 30.Nash, J.: Non-cooperative games. Ann. Math.
**54**(2), 286–295 (1951)MathSciNetCrossRefzbMATHGoogle Scholar - 31.Orgeta, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations in Several Variables. Classics in Applied Mathematics. SIAM Publications, Philadelphia (2000)Google Scholar
- 32.Ovchinnikov, S.: Max-min representation of piecewise linear functions. Contrib. Algebra Geom.
**43**, 297–302 (2002)MathSciNetzbMATHGoogle Scholar - 33.Pang, J.S., Tao, M.: Decomposition methods for computing directional stationary solutions of a class of non-smooth nonconvex optimization problems. SIAM J. Optim. Submitted January (2017)Google Scholar
- 34.Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth dc programs. Math. Oper. Res.
**42**, 95–118 (2017)MathSciNetCrossRefzbMATHGoogle Scholar - 35.Pang, J.-S., Sen, S., Shanbhag, U.V.: Two-stage non-cooperative games with risk-averse players. Math. Program.
**165**(1), 235–290 (2017)MathSciNetCrossRefzbMATHGoogle Scholar - 36.Pham, D.T., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithm and applications. Acta Math. Vietnam.
**22**, 289–355 (1997)MathSciNetzbMATHGoogle Scholar - 37.Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: Parallel successive convex approximation for nonsmooth nonconvex optimization. Adv. Neural Inf. Process. Syst. (NIPS), 1440–1448 (2014)Google Scholar
- 38.Razaviyayn, M.: Successive Convex Approximations: Analysis and Applications. Ph.D. Dissertation. Department of Electrical and Computer Engineering. University of Minnesota (Minneapolis 2014)Google Scholar
- 39.Razaviyayn, M., Hong, M., Luo, Z.Q.: A unified convergence analysis of block successive minimization methods for nonsmooth optimization. SIAM J. Optim.
**23**, 1126–1153 (2013)MathSciNetCrossRefzbMATHGoogle Scholar - 40.Razaviyayn, M., Hong, M., Luo, Z.Q., Pang, J.S.: A unified algorithmic framework for block-structured optimization involving big data. IEEE Signal Process. Mag.
**33**, 57–77 (2016)CrossRefGoogle Scholar - 41.Razaviyayn, M., Sanjabi, M., Luo, Z.Q.: A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks. Math. Program. Ser. B
**157**, 515–545 (2016)MathSciNetCrossRefzbMATHGoogle Scholar - 42.Rockafellar, R.T., Uryasev, S.: Optimization of conditional value-at-risk. J. Risk
**2**, 21–41 (2000)CrossRefGoogle Scholar - 43.Rockafellar, R.T., Uryasev, S.: Conditional value-at-risk for general loss distributions. J. Bank. Finance
**7**, 1143–1471 (2002)Google Scholar - 44.Rockafellar, R.T., Uryasev, S.: The fundamental risk quadrangle in risk management, optimization, and statistical estimation. Surv. Oper. Res. Manag. Sci.
**18**, 33–53 (2013)MathSciNetGoogle Scholar - 45.Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Optimality conditions in portfolio analysis with general deviation measures. Math. Program. Ser. B
**108**, 515–540 (2006)MathSciNetCrossRefzbMATHGoogle Scholar - 46.Rockafellar, R.T., Uryasev, S., Zabarankin, M.: Generalized deviations in risk analysis. Finance Stochast.
**10**, 51–74 (2006)MathSciNetCrossRefzbMATHGoogle Scholar - 47.Sarykalin, S., Uryasev, S.: Value-at-risk versus conditional value-at-risk in risk management and optimization. Tutor. Oper. Res.
**2008**, 269–294 (2008)Google Scholar - 48.Scholtes, S.: Intoduction to Piecewise Differentiable Functions. Springer, Berlin (2012)CrossRefzbMATHGoogle Scholar
- 49.Scutari, G., Facchinei, F., Palomar, D.P., Pang, J.S., Song, P.: Decomposition by partial linearization: parallel optimization of multiuser systems. IEEE Trans. Signal Process.
**62**, 641–656 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 50.Scutari, G., Alvarado, A., Pang, J.S.: A new decomposition method for multiuser DC-programming and its application to physical layer security. IEEE Trans. Signal Process.
**62**, 2984–2998 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 51.Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM Publications, Philadelphia (2009)CrossRefzbMATHGoogle Scholar
- 52.Sun, J.: On the structure of convex piecewise quadratic functions. J. Optim. Theory Appl.
**72**, 499–510 (1992)MathSciNetCrossRefzbMATHGoogle Scholar - 53.Tuy, H.: Convex Analysis and Global Optimization, 2nd edn, vol. 110. Springer Optimization and its Applications (2016). [First edition. Kluwer Publishers (Dordrecht 1998)]Google Scholar
- 54.Tuy, H.: Global minimization of a difference of two convex functions. Math. Program. Study
**30**, 150–187 (1987)MathSciNetCrossRefzbMATHGoogle Scholar - 55.Veselý, L., Zajíček, L.: Delta-convex mappings between Banach spaces and applications. Dissertationes Math. (Rozprawy Mat.) 289 (1989)Google Scholar
- 56.Veselý, L., Zajíček, L.: On composition of DC functions and mappings. J. Conv. Anal.
**16**(2), 423–439 (2009)zbMATHGoogle Scholar - 57.Wozabal, D.: Value-at-risk optimization using the difference of convex algorithm. OR Spectr.
**34**, 861–883 (2012)MathSciNetCrossRefzbMATHGoogle Scholar