Mathematical Programming

, Volume 152, Issue 1–2, pp 435–466 | Cite as

A framework of discrete DC programming by discrete convex analysis

Full Length Paper Series A

Abstract

A theoretical framework of difference of discrete convex functions (discrete DC functions) and optimization problems for discrete DC functions is established. Standard results in continuous DC theory are exported to the discrete DC theory with the use of discrete convex analysis. A discrete DC algorithm, which is a discrete analogue of the continuous DC algorithm (Concave–Convex procedure in machine learning) is proposed. The algorithm contains the submodular-supermodular procedure as a special case. Exploiting the polyhedral structure of discrete convex functions, the algorithms tailored to specific types of discrete DC functions are proposed.

Mathematics Subject Classification

90C27 90C46 

References

  1. 1.
    Bačák, M., Borwein, J.M.: On difference convexity of locally Lipschitz functions. Optimization 60, 961–978 (2011)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Birkhoff, G.: Rings of sets. Duke Math. J. 3, 443–454 (1937)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Calinescu, G., Chekuri, C., Pál, M., Vondrák, J.: Maximizing a monotone submodular function subject to a matroid constraint. SIAM J. Comput. 40, 1740–1766 (2011)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Dress, A.W.M., Wenzel, W.: Valuated matroids. Adv. Math. 93, 214–250 (1992)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Edmonds, J.: Submodular functions, matroids and certain polyhedra. In: Guy, R., Hanani, H., Sauer, N., Schönheim, J. (eds.) Combinatorial Structures and Their Applications, pp. 69–87. Gordon and Breach, New York (1970)Google Scholar
  6. 6.
    Favati, P., Tardella, F.: Convexity in nonlinear integer programming. Ricerca Operat. 53, 3–44 (1990)Google Scholar
  7. 7.
    Feige, U., Mirrokni, V.S., Vondrák, J.: Maximizing non-monotone submodular functions. SIAM J. Comput. 40, 1133–1153 (2011)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Frank, A.: Generalized polymatroids. In: Hajnal, A., Lovász, L., Sós, V. T. (eds.) Finite and Infinite Sets, (Proceedings of 6th Hungarian Combinatorial Colloquium, 1981), Colloquia Mathematica Societatis János Bolyai, vol. 37, pp. 285–294, North-Holland (1984)Google Scholar
  9. 9.
    Frank, A.: Connections in Combinatorial Optimization. Oxford Lecture Series in Mathematics and its Applications, 38, Oxford University Press, Oxford (2011)Google Scholar
  10. 10.
    Frank, A., Tardos, É.: Generalized polymatroids and submodular flows. Math. Program. 42, 489–563 (1988)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Fujishige, S.: Submodular Functions and Optimization, 2nd edn. Annals of Discrete Mathematics, vol. 58. Elsevier, Amsterdam (2005)Google Scholar
  12. 12.
    Fujishige, S., Murota, K.: Notes on L-/M-convex functions and the separation theorems. Math. Program. 88, 129–146 (2000)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Hartman, P.: On functions representable as a difference of convex functions. Pac. J. Math. 9, 707–713 (1959)CrossRefGoogle Scholar
  14. 14.
    Hirai, H., Murota, K.: M-convex functions and tree metric. Jpn. J. Ind. Appl. Math. 21, 391–403 (2004)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hiriart-Urruty, J.B.: Generalized differentiability, duality and optimization for problems dealing with differences of convex functions. Lecture Note Econ. Math. Syst. 256, 37–70 (1985)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Hiriart-Urruty, J.B.: From convex optimization to nonconvex optimization, Part I: Necessary and sufficient conditions for global optimality. In: Clarke, F.H., Dem’yanov, V.F., Giannessi, F. (eds.) Nonsmooth Optimization and Related Topics, Ettore Majorana International Sciences, Physical Sciences, Series 43, pp. 219–239. Plenum Press, New York (1989)Google Scholar
  17. 17.
    Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms I. Springer, Berlin (1993)Google Scholar
  18. 18.
    Horst, R., Thoai, N.V.: DC Programming: overview. J. Opt. Theory Appl. 103, 1–43 (1999)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Iwata S., Orlin, J.B.: A simple combinatorial algorithm for submodular function minimization. In: Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms, Society for Industrial and Applied Mathematics, pp. 1230–1237 (2009)Google Scholar
  20. 20.
    Iyer, R., Bilmes, J.: Algorithms for approximate minimization of the difference between submodular functions, with applications. In: Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence, pp. 407–417. Also: http://arxiv.org/abs/1207.0560 (2012)
  21. 21.
    Iyer, R., Jegelka, S., Bilmes, J.: Fast semidifferential-based submodular function optimization. In: Proceedings of the 30th International Conference on Machine Learning, pp. 855–863 (2013)Google Scholar
  22. 22.
    Kawahara, Y., Nagano, K., Okamoto, Y.: Submodular fractional programming for balanced clustering. Pattern Recogn. Lett. 32, 235–243 (2011)CrossRefGoogle Scholar
  23. 23.
    Kawahara, Y., Washio, T.: Prismatic algorithm for discrete D.C. programming problem. In: Proceedings of the 25th Annual Conference on Neural Information Processing Systems, pp. 2106–2114 (2011)Google Scholar
  24. 24.
    Kolmogorov, V., Shioura, A.: New algorithms for convex cost tension problem with application to computer vision. Discret. Optim. 6, 378–393 (2009)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Lee, J., Mirrokni, V.S., Nagarajan, V., Sviridenko, M.: Non-monotone submodular maximization under matroid and knapsack constraints. In: Proceedings of the 41st Annual ACM Symposium on Theory of Computing, pp. 323–332 (2009)Google Scholar
  26. 26.
    McCormick, S.T.: Submodular containment is hard, even for networks. Oper. Res. Lett. 19, 95–99 (1996)MathSciNetCrossRefGoogle Scholar
  27. 27.
    McCormick, S.T.: Submodular function minimization. In: Aardal, K., Nemhauser, G., Weismantel, R. (eds.) Discrete Optimization, Handbooks in Operations Research and Management Science, Chap. 7, vol. 12, pp. 321–391. Elsevier Science Publishers, Berlin (2006)Google Scholar
  28. 28.
    Miller, B.L.: On minimizing nonseparable functions defined on the integers with an inventory applications. SIAM J. Appl. Math. 21, 166–185 (1971)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Moriguchi, S., Murota, K.: Discrete Hessian matrix for L-convex functions. In: IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, E83-A, pp. 1104–1108 (2005)Google Scholar
  30. 30.
    Moriguchi, S., Murota, K.: On discrete Hessian matrix and convex extensibility. J. Oper. Res. Soc. Jpn. 55, 48–62 (2012)MathSciNetGoogle Scholar
  31. 31.
    Murota, K.: Valuated matroid intersection, I: optimality criteria. SIAM J. Discret. Math. 9, 545–561 (1996)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Murota, K.: Valuated matroid intersection, II: algorithms. SIAM J. Discret. Math. 9, 562–576 (1996)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Murota, K.: Matroid valuation on independent sets. J. Combinatorial Theory Ser. B 69, 59–78 (1997)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Murota, K.: Discrete convex analysis. Math. Program. 83, 313–371 (1998)MathSciNetGoogle Scholar
  35. 35.
    Murota, K.: Matrices and Matroids for Systems Analysis. Springer, Berlin (2000)Google Scholar
  36. 36.
    Murota, K.: Discrete Convex Analysis. Society for Industrial and Applied Mathematics, Philadelphia (2003)CrossRefGoogle Scholar
  37. 37.
    Murota, K.: Recent developments in discrete convex analysis. In: Cook, W., Lovász, L., Vygen, J. (eds.) Research Trends in Combinatorial Optimization, Chap. 11, pp. 219–260. Springer, Berlin (2009)CrossRefGoogle Scholar
  38. 38.
    Murota, K., Shioura, A.: M-convex function on generalized polymatroid. Math. Oper. Res. 24, 95–105 (1999)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Murota, K., Shioura, A.: Exact bounds for steepest descent algorithms of L-convex function minimization. Oper. Res. Lett. to appear (2014)Google Scholar
  40. 40.
    Narasimhan, M., Bilmes, J.: A submodular-supermodular procedure with applications to discriminative structure learning. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, pp. 404–412 (2005)Google Scholar
  41. 41.
    Onn, S.: Nonlinear Discrete Optimization: An Algorithmic Theory. European Mathematical Society, Zurich (2010)CrossRefGoogle Scholar
  42. 42.
    Orlin, J.B.: A faster strongly polynomial time algorithm for submodular function minimization. Math. Program. Ser. A 118, 237–251 (2009)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Queyranne, M., Tardella, F.: Submodular function minimization in \(\mathbb{Z}^n\) and searching in Monge arrays. Unpublished manuscript, Presented by M. Queyranne at CTW04 (Cologne Twente Workshop 2004), Electronic Notes in Discrete Mathematics, vol. 17, p. 5 (2004)Google Scholar
  44. 44.
    Schrijver, A.: Combinatorial Optimization: Polyhedra and Efficiency. Springer, Berlin (2003)Google Scholar
  45. 45.
    Singer, I.: A Fenchel-Rockafellar type duality theorem for maximization. Bull. Aust. Math. Soc. 20, 193–198 (1979)CrossRefGoogle Scholar
  46. 46.
    Tao, P.D., El Bernoussi, S.: Duality in D.C. (difference of convex functions) optimization: Subgradient methods. In: Hoffman, K.H., Zowe, J., Hiriart-Urruty, J.B., Lemaréchal, C. (eds.) Trends in Mathematical Optimization, International Series of Numerical Mathematics, vol. 84, pp. 277–293. Basel, Birkhäuser (1987)Google Scholar
  47. 47.
    Tao, P.D.: Hoai An, L.T.: Convex analysis approach to D.C. programming: Theory, algorithms and applications. Acta Math. Vietnam. 22, 289–355 (1997)MathSciNetGoogle Scholar
  48. 48.
    Toland, J.F.: A duality principle for non-convex optimisation and the calculus of variations. Arch. Ration. Mech. Anal. 71, 41–61 (1979)MathSciNetCrossRefGoogle Scholar
  49. 49.
    Topkis, D.M.: Minimizing a submodular function on a lattice. Oper. Res. 26, 305–321 (1978)MathSciNetCrossRefGoogle Scholar
  50. 50.
    Tuy, H.: Global minimization of a difference of two convex functions. Math. Program. Study 30, 150–182 (1987)CrossRefGoogle Scholar
  51. 51.
    Tuy, H.: D.C. optimization: Theory, methods and algorithms. In: Horst, R., Pardalos, P.M. (eds.) Handbook of Global Optimization, pp. 149–216. Kluwer Academic Publishers, Dordrecht (1995)Google Scholar
  52. 52.
    Vondrák, J.: Optimal approximation for the submodular welfare problem in the value oracle model. In: Proceedings of the 40th Annual ACM Symposium on Theory of Computing, pp. 67–74 (2008)Google Scholar
  53. 53.
    Yuille, A.L., Rangarajan, A.: The concave-convex procedure. Neural Comput. 15, 915–936 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and Mathematical Optimization Society 2014

Authors and Affiliations

  1. 1.JST, ERATO, Kawarabayashi ProjectNational Institute of InformaticsTokyoJapan
  2. 2.Department of Mathematical InformaticsUniversity of TokyoTokyoJapan

Personalised recommendations