Advertisement

Nonsmooth and Nonconvex Optimization via Approximate Difference-of-Convex Decompositions

  • Wim van Ackooij
  • Welington de OliveiraEmail author
Article
First Online:

Abstract

We propose an optimization technique for computing stationary points of a broad class of nonsmooth and nonconvex programming problems. The proposed approach (approximately) decomposes the objective function as the difference of two convex functions and performs inexact optimization of the resulting (convex) subproblems. We prove global convergence of our method in the sense that, for an arbitrary starting point, every accumulation point of the sequence of iterates is a Clarke-stationary solution. The given approach is validated by encouraging numerical results on several nonsmooth and nonconvex distributionally robust optimization problems.

Keywords

Nonconvex programming Nonsmooth optimization Lower-\(C^2\) functions DC decomposition 

Mathematics Subject Classification

49J52 49J53 49K99 90C26 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The authors would like to acknowledge financial support from the Gaspard-Monge program for Optimization and Operations Research (PGMO) project “Optimization & stability of stochastic unit-commitment problems.”

References

  1. 1.
    Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9(3), 167–198 (1959)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Rockafellar, R., Wets, R.J.B.: Variational Analysis, Grundlehren der mathematischen Wissenschaften, vol. 317, 3rd edn. Springer, Berlin (2009)Google Scholar
  3. 3.
    Pflug, G.C., Pohl, M.: A review on ambiguity in stochastic portfolio optimization. Set Valued Var. Anal. (2017).  https://doi.org/10.1007/s11228-017-0458-z zbMATHCrossRefGoogle Scholar
  4. 4.
    Esfahani, P.M., Kuhn, D.: Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Math. Program. 171(1–2), 115–166 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Curtis, F.E., Mitchell, T., Overton, M.L.: A BFGS-SQP method for nonsmooth, nonconvex, constrained optimization and its evaluation using relative minimization profiles. Optim. Methods Softw. 32(1), 148–181 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Ginchev, I., Gintcheva, D.: Characterization and recognition of D.C. functions. J. Glob. Optim. 57(3), 633–647 (2012).  https://doi.org/10.1007/s10898-012-9964-6 MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Toa, P.D., Souad, E.B.: Duality in D.C. (difference of convex functions) optimization. Subgradient methods. In: Hoffmann, K.H., Zowe, J., Hiriart-Urruty, J., Lemarechal, C. (eds.) Trends in Mathematical Optimization. International Series of Numerical Mathematics, vol. 84, pp. 277–293. Birkhauser, Basel (1988)Google Scholar
  8. 8.
    de Oliveira, W., Tcheou, M.: An inertial algorithm for DC programming. Set Valued Var. Anal. (2018).  https://doi.org/10.1007/s11228-018-0497-0
  9. 9.
    Toa, P.D.: Exact penalty in D.C. programming. Vietnam J. Math. 27(2), 169–178 (1999)MathSciNetGoogle Scholar
  10. 10.
    Tao, P.D., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnam. 22(1), 289–355 (1997)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017).  https://doi.org/10.1287/moor.2016.0795 MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Strekalovsky, A.S.: On local search in D.C. optimization problems. Appl. Math. Comput. 255(1), 73–83 (2015)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Strekalovsky, A.S., Minarchenko, I.M.: A local search method for optimisation problem with D.C. inequality constraints. Appl. Math. Model. 1(58), 229–244 (2018)CrossRefGoogle Scholar
  14. 14.
    Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes. J. Glob. Optim. 68(3), 501–535 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Glob. Optim. 71, 37–55 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    de Oliveira, W.: Proximal bundle methods for nonsmooth DC programming. J. Glob. Optim. (2019).  https://doi.org/10.1007/s10898-019-00755-4 CrossRefMathSciNetzbMATHGoogle Scholar
  17. 17.
    Le Thi, H.A., Tao, P.D.: DC programming in communication systems: challenging problems and methods. Vietnam J. Comput. Sci. 1(1), 15–28 (2014).  https://doi.org/10.1007/s40595-013-0010-5 CrossRefGoogle Scholar
  18. 18.
    Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math. Program. 116(1–2), 221–258 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Hare, W., Sagastizábal, C., Solodov, M.: A proximal bundle method for nonconvex functions with inexact oracles. Comput. Optim. Appl. 63(1), 1–28 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Karmitsa, N., Gaudioso, M., Joki, K.: Diagonal bundle method with convex and concave updates for large-scale nonconvex and nonsmooth optimization. Optim. Methods Softw. (2017).  https://doi.org/10.1080/10556788.2017.1389941 zbMATHCrossRefGoogle Scholar
  22. 22.
    Dao, M.N., Gwinner, J., Noll, D., Ovcharova, N.: Nonconvex bundle method with application to a delamination problem. Comput. Optim. Appl. 65(1), 173–203 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Fuduli, A., Gaudioso, M., Giallombardo, G.: Minimizing nonconvex nonsmooth functions via cutting planes and proximity control. SIAM J. Optim. 14(3), 743–756 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Fuduli, A., Gaudioso, M., Giallombardo, G.: A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization. Optim. Methods Softw. 19, 89–102 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Haarala, N., Miettinen, K., Mäkelä, M.M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109(1), 181–205 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Kiwiel, K.C.: A linearization algorithm for nonsmooth minimization. Math. Oper. Res. 10, 185–194 (1985)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Kiwiel, K.: Restricted step and Levenberg–Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6(1), 227–249 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    Lukšan, L., Vlček, J.: A bundle-newton method for nonsmooth unconstrained minimization. Math. Program. 83(1–3), 373–391 (1998)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. World Scientific Publishing Co., River Edge (1992)zbMATHCrossRefGoogle Scholar
  30. 30.
    Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2, 121–152 (1992)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained optimization. J. Optim. Theory Appl. 111, 407–430 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Mifflin, R.: A modification and extension of Lemaréchal’s algorithm for nonsmooth optimization. Math. Program. Study 17(1), 77–90 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Math. Program. 69(1), 111–147 (1995)MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    van Ackooij, W., de Oliveira, W.: Level bundle methods for constrained convex optimization with various oracles. Comput. Optim. Appl. 57(3), 555–597 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Noll, D.: Bundle method for non-convex minimization with inexact subgradients and function values. In: Bailey, D.H., Bauschke, H.H., Borwein, P., Garvan, F., Théra, M., Vanderwerff, J., Wolkowicz, H. (eds.) Computational and Analytical Mathematics. Springer Proceedings in Mathematics and Statistics, vol. 50, pp. 555–592. Springer, Berlin (2013)CrossRefzbMATHGoogle Scholar
  36. 36.
    Luc, D.T., Van Ngai, H., Théra, M.: On \(\epsilon \)-monotonicity and \(\epsilon \)-convexity. In: Ioffe, A., Reich, S., Shafrir, I. (eds.) Calculus of Variations and Differential Equations. Research Notes in Mathematics, vol. 410, pp. 82–100. Chapman and Hall/CRC, London (2000)zbMATHGoogle Scholar
  37. 37.
    Daniildis, A., Georgiev, P.: Approximate convexity and submonotonicity. J. Math. Anal. Appl. 291, 117–144 (2004)MathSciNetGoogle Scholar
  38. 38.
    Apkarian, P., Noll, D., Prot, O.: A proximity control algorithm to minimize nonsmooth and nonconvex semi-infinite maximum eigenvalue functions. J. Convex Anal. 16(3–4), 641–666 (2009)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Clarke, F.: Optimisation and nonsmooth analysis. Classics in applied mathematics. Soc. Ind. Appl. Math. (1987).  https://doi.org/10.1137/1.9781611971309 CrossRefGoogle Scholar
  40. 40.
    Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I. Basic Theory. Grundlehren der mathematischen Wissenschaften, vol. 330. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  41. 41.
    Borwein, J.M., Preiss, D.: A smooth variational principle with applications to subdifferentiability and differentiability of convex functions. Trans. Am. Math. Soc. 303, 517–527 (1987)MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    Borwein, J.M., Zhu, Q.J.: Techniques of Variational Analysis, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 20. Springer, New York (2005)Google Scholar
  43. 43.
    Kruger, A.Y.: On Fréchet subdifferentials. J. Math. Sci. 116(3), 3325–3358 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Kelley, J.E.: The cutting-plane method for solving convex programs. J. Soc. Ind. Appl. Math. 8(4), 703–712 (1960)MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Frangioni, A.: Standard Bundle Methods: Untrusted Models and Duality. Technical Report del Dipartimento di Informatica, TR. University of Pisa, Pisa, IT (submitted)Google Scholar
  46. 46.
    de Oliveira, W.: Target radius methods for nonsmooth convex optimization. Oper. Res. Lett. 45(6), 659–664 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  47. 47.
    Bello-Cruz, J.Y., de Oliveira, W.: Level bundle-like algorithms for convex optimization. J. Glob. Optim. 59(4), 787–809 (2014).  https://doi.org/10.1007/s10898-013-0096-4 MathSciNetzbMATHCrossRefGoogle Scholar
  48. 48.
    Ferrier, C.: Bornes duales de problèmes d’optimisation polynomiaux. Ph.D. thesis, Université Paul Sabatier, Toulouse (1997)Google Scholar
  49. 49.
    Ferrier, C.: Computation of the distance to semi-algebraic sets. ESAIM Control Optim. Calc. Var. 5(1), 139–156 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  50. 50.
    Beltran, F., de Oliveira, W., Finardi, E.C.: Application of scenario tree reduction via quadratic process to medium-term hydrothermal scheduling problem. IEEE Trans. Power Syst. 32(6), 4351–4361 (2017)CrossRefGoogle Scholar
  51. 51.
    Trivedi, P.K., Zimmer, D.M.: Copula modeling: an introduction for practitioners. Found. Trends Econ. 1(1), 1–111 (2007).  https://doi.org/10.1561/0800000005 zbMATHCrossRefGoogle Scholar
  52. 52.
    Nelsen, R.B.: An Introduction to Copulas. Springer Series in Statistics, 2nd edn. Springer, New York (2006)zbMATHGoogle Scholar
  53. 53.
    McNeil, A., Nešlehová, J.: Multivariate archimedian copulas, d-monotone functions and \(l_1\) norm symmetric distributions. Ann. Stat. 37, 3059–3097 (2009)zbMATHCrossRefGoogle Scholar
  54. 54.
    van Ackooij, W., de Oliveira, W.: Convexity and optimization with copulæ structured probabilistic constraints. Optim. J. Math. Program. Oper. Res. 65(7), 1349–1376 (2016).  https://doi.org/10.1080/02331934.2016.1179302 zbMATHCrossRefGoogle Scholar
  55. 55.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002).  https://doi.org/10.1007/s101070100263 MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.EDF R&DÉlectricité de FranceParisFrance
  2. 2.MINES ParisTech, PSL – Research University, CMA – Centre de Mathématiques AppliquéesSophia AntipolisFrance

Personalised recommendations

Cite article

Buy options