Advertisement

Set-Valued and Variational Analysis

, Volume 27, Issue 4, pp 895–919 | Cite as

An Inertial Algorithm for DC Programming

  • Welington de OliveiraEmail author
  • Michel P. Tcheou
Article

Abstract

We consider nonsmooth optimization problems whose objective function is defined by the Difference of Convex (DC) functions. With the aim of computing critical points that are also d(irectional)-stationary for such a class of nonconvex programs we propose an algorithmic scheme equipped with an inertial-force procedure. In contrast to the classical DC algorithm of P. D. Tao and L. T. H. An, the proposed inertial DC algorithm defines trial points whose sequence of functional values is not necessary monotonically decreasing, a property that proves useful to prevent the algorithm from converging to a critical point that is not d-stationary. Moreover, our method can handle inexactness in the solution of convex subproblems yielding trial points. This is another property of practical interest that substantially reduces the computational burden to compute d-stationary/critical points of DC programs. Convergence analysis of the proposed algorithm yields global convergence to critical points, and convergence rate is established for the considered class of problems. Numerical experiments on large-scale (nonconvex and nonsmooth) image denoising models show that the proposed algorithm outperforms the classic one in this particular application, specifically in the case of piecewise constant images with neat edges such as QR codes.

Keywords

DC programming Nonsmooth optimization Variational analysis 

Mathematics Subject Classification (2010)

49J52 49J53 49K99 90C26 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgments

The authors are grateful to the Reviewers and the Associate Editor for their remarks and constructive suggestions that considerably improved the original version of this article.

References

  1. 1.
    Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19 (9), 2345–2356 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Alvarez, F.: Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in hilbert space. SIAM J. Optim. 14 (3), 773–782 (2004)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Alvarez, F., Attouch, H.: An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping. Set-Val. Anal. 9(1), 3–11 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    An, L.T.H., Tao, P.D.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1), 23–46 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Bagirov, A.M., Yearwood, J.: A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170(2), 578–596 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Beck, A., Teboulle, M.: Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process. 18 (11), 2419–2434 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Bello Cruz, J.Y.: On proximal subgradient splitting method for minimizing the sum of two nonsmooth convex functions. Set-Val. Var. Anal. 25(2), 245–263 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Caselles, V., Chambolle, A., Cremers, D., Novaga, M., Pock, T.: An introduction to total variation for image analysis. Theoretical Foundations and Numerical Methods for Sparse Recovery, De Gruyter, Radon Series. Comp. Appl. Math. 9, 263–340 (2010)zbMATHGoogle Scholar
  9. 9.
    Condat, L.: A primal–dual splitting method for convex optimization involving lipschitzian, proximable and linear composite terms. J. Optim. Theory Appl. 158(2), 460–479 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    de Oliveira, W.: Proximal Bundle Methods for Nonsmooth DC Programming. Technical report. Available at www.oliveira.mat.br (2017)
  12. 12.
    Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Gaudioso, M., Giallombardo, G., Miglionico, G.: Minimizing piecewise-concave functions over polyhedra. Math. Oper. Res. 43(2), 580–597 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Glob. Optim. 71, 37–55 (2018)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice-Hall, Inc., Upper Saddle River (2006)Google Scholar
  16. 16.
    Gruzdeva, T., Strekalovsky, A.: A D.C. Programming Approach to Fractional Problems, pp 331–337. Springer International Publishing, Cham (2017)zbMATHGoogle Scholar
  17. 17.
    Hiriart-Urruty, J.: Optimisation et analyse convexe. Presses Universitaires de France, France (1998)zbMATHGoogle Scholar
  18. 18.
    Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I, 2nd edn. No. 305 in Grundlehren der mathematischen Wissenschaften. Springer, Berlin (1996)Google Scholar
  19. 19.
    Hiriart-Urruty, J.B.: Generalized Differentiability/Duality and Optimization for Problems Dealing with Differences of Convex Functions, pp 37–70. Springer, Berlin (1985)zbMATHGoogle Scholar
  20. 20.
    Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305-306 in Grund. der math. Wiss. Springer, Berlin (1993). (two volumes)CrossRefGoogle Scholar
  21. 21.
    Holmberg, K., Tuy, H.: A production-transportation problem with stochastic demand and concave production costs. Math. Program. 85(1), 157–179 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Izmailov, A.F., Solodov, M.V.: Newton-type Methods for Optimization and Variational Problems, 1st edn. Springer Series in Operations Research and Financial Engineering Springer International Publishing (2014)Google Scholar
  23. 23.
    Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth dc optimization utilizing nonconvex cutting planes. J. Glob. Optim. 68(3), 501–535 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Khalaf, W., Astorino, A., d’Alessandro, P., Gaudioso, M.: A DC optimization-based clustering technique for edge detection. Optim. Lett. 11(3), 627–640 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Lanza, A., Morigi, S., Sgallari, F.: Convex image denoising via non-convex regularization with parameter selection. J. Math. Imaging Vis. 56(2), 195–220 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Le Thi, H.A., Pham Dinh, T.: Dc programming in communication systems: challenging problems and methods. Vietnam J. Comput. Sci. 1(1), 15–28 (2014)CrossRefGoogle Scholar
  27. 27.
    Le Thi, H.A., Pham Dinh, T., Ngai, H.V.: Exact penalty and error bounds in dc programming. J. Glob. Optim. 52(3), 509–535 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  28. 28.
    w. Lu, C.: Image restoration and decomposition using nonconvex non-smooth regularisation and negative hilbert-sobolev norm. IET Image Process. 6(6), 706–716 (2012)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Moudafi, A., Oliny, M.: Convergence of a splitting inertial proximal method for monotone operators. J. Comput. Appl. Math. 155(2), 447–454 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7(2), 1388–1419 (2014)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)MathSciNetzbMATHCrossRefGoogle Scholar
  33. 33.
    Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)CrossRefGoogle Scholar
  34. 34.
    Rockafellar, R.: Convex Analysis, 1st edn. Princeton University Press, Princeton (1970)Google Scholar
  35. 35.
    Souza, J.C.O., Oliveira, P.R., Soubeyran, A.: Global convergence of a proximal linearized algorithm for difference of convex functions. Optim. Lett. 10(7), 1529–1539 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Strekalovskiy, A.: An exact penalty method for d.c. optimization. AIP Conf. Proc. 1776(1), 060,003 (2016)CrossRefGoogle Scholar
  37. 37.
    Strekalovsky, A., Minarchenko, I.: On Local Search in d.c. Optimization. In: 2017 Constructive Nonsmooth Analysis and Related Topics (Dedicated to the Memory of V.F. Demyanov) (CNSA), pp. 1–4 (2017)Google Scholar
  38. 38.
    Strekalovsky, A.S.: On local search in d.c. optimization problems. Appl. Math. Comput. 255(1), 73–83 (2015)MathSciNetzbMATHGoogle Scholar
  39. 39.
    Tao, P.D., An, L.T.H.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnamica 22(1), 289–355 (1997)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Tao, P.D., Souad, E.B.: Algorithms for solving a class of nonconvex optimization problems. methods of subgradients. North-Holland Math. Stud. 129, 249–271 (1986). Fermat Days 85: Mathematics for OptimizationMathSciNetzbMATHCrossRefGoogle Scholar
  41. 41.
    Tuy, H.: Convex Analysis and Global Optimization, 2nd edn. Springer Optimization and Its Applications. Springer (2016)Google Scholar
  42. 42.
    van Ackooij, W., de Oliveira, W.: DC Programming Techniques with Inexact Subproblems’ Solution for General DC Programs. Technical Report, CMA. Available at www.oliveira.mat.br (2017)
  43. 43.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  44. 44.
    Zavriev, S.K., Kostyuk, F.V.: Heavy-ball method in nonconvex optimization problems. Comput. Math. Model. 4(4), 336–341 (1993)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2018

Authors and Affiliations

  1. 1.MINES ParisTechPSL – Research University, CMA – Centre de Mathématiques AppliquéesSophia AntipolisFrance
  2. 2.Rio de Janeiro State University (UERJ)Rio de JaneiroBrazil

Personalised recommendations