Skip to main content
SpringerLink
Log in

Classes of linear programs solvable by coordinate-wise minimization

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Coordinate-wise minimization is a simple popular method for large-scale optimization. Unfortunately, for general (non-differentiable and/or constrained) convex problems, its fixed points may not be global minima. We present two classes of linear programs (LPs) that coordinate-wise minimization solves exactly. We show that these classes subsume the dual LP relaxations of several well-known combinatorial optimization problems and the method finds a global minimum with sufficient accuracy in reasonable runtimes. Moreover, we experimentally show that the method frequently yields good suboptima or even optima for sparse LPs where optimality is not guaranteed in theory. Though the presented problems can be solved by more efficient methods, our results are theoretically non-trivial and can lead to new large-scale optimization algorithms in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ansotegui, C., Bacchus, F., Järvisalo, M., Martins, R., et al.: MaxSAT Evaluation 2017. https://helda.helsinki.fi/bitstream/handle/10138/233112/mse17proc.pdf (2017)

  2. Bacchus, F., Järvisalo, M., Martins, R.: MaxSAT evaluation 2018: New developments and detailed results. Journal on Satisfiability, Boolean Modeling and Computation 11(1), 99–131 (2019). Instances available at https://maxsat-evaluations.github.io/

    Article  MathSciNet  Google Scholar 

  3. Bacchus, F., Järvisalo, M.J., Martins, R., et al.: MaxSAT Evaluation 2018. https://helda.helsinki.fi/bitstream/handle/10138/237139/mse18_proceedings.pdf (2018)

  4. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Belmont, Athena Scientific (1999)

    MATH  Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning 3(1), 1–122 (2011)

    Article  Google Scholar 

  6. Boykov, Y., Lempitsky, V.S.: From photohulls to photoflux optimization. In: Proceedings of the British Machine Conference, vol. 3, p 27. Citeseer (2006)

  7. Boykov, Y., Veksler, O., Zabih, R.: Markov random fields with efficient approximations. In: Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. 98CB36231), pp. 648–655. IEEE (1998)

  8. Brualdi, R.A.: Combinatorial Matrix Classes, vol. 13. Cambridge University Press, Cambridge (2006)

    Book  Google Scholar 

  9. Carpaneto, G., Toth, P.: Primal-dual algrorithms for the assignment problem. Discret. Appl. Math. 18(2), 137–153 (1987)

    Article  Google Scholar 

  10. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision 40(1), 120–145 (2011)

    Article  MathSciNet  Google Scholar 

  11. Chudak, F.A., Hochbaum, D.S.: A half-integral linear programming relaxation for scheduling precedence-constrained jobs on a single machine. Oper. Res. Lett. 25(5), 199–204 (1999)

    Article  MathSciNet  Google Scholar 

  12. Dechter, R.: Tractable structures for constraint satisfaction problems. In: Handbook of Constraint Programming, chap. 7. Elsevier (2006)

  13. Dell’Amico, M., Toth, P.: Algorithms and codes for dense assignment problems: the state of the art. Discret. Appl. Math. 100(1-2), 17–48 (2000)

    Article  MathSciNet  Google Scholar 

  14. Dlask, T.: Unit propagation by means of coordinate-wise minimization. In: International Conference on Machine Learning, Optimization, and Data Science. Springer (2020)

  15. Dlask, T., Werner, T.: Bounding linear programs by constraint propagation: Application to Max-SAT. In: International Conference on Principles and Practice of Constraint Programming, pp. 177–193. Springer (2020)

  16. Dlask, T., Werner, T.: A class of linear programs solvable by coordinate-wise minimization. In: Kotsireas, I.S., Pardalos, P.M. (eds.) Learning and Intelligent Optimization, pp. 52–67. Springer (2020)

  17. Dlask, T., Werner, T.: On relation between constraint propagation and block-coordinate descent in linear programs. In: International Conference on Principles and Practice of Constraint Programming, pp. 194–210. Springer (2020)

  18. Duan, R., Pettie, S.: Linear-time approximation for maximum weight matching. Journal of the ACM (JACM) 61(1), 1–23 (2014)

    Article  MathSciNet  Google Scholar 

  19. Edmonds, J.: Maximum matching and a polyhedron with 0, 1-vertices. J. Res. Natl. Bur. Stand. B 69(125-130), 55–56 (1965)

    MathSciNet  Google Scholar 

  20. Edmonds, J., Karp, R.M.: Theoretical improvements in algorithmic efficiency for network flow problems. Journal of the ACM (JACM) 19(2), 248–264 (1972)

    Article  Google Scholar 

  21. Ferdous, S., Pothen, A., Khan, A.: New approximation algorithms for minimum weighted edge cover. In: 2018 Proceedings of the Seventh SIAM Workshop on Combinatorial Scientific Computing, pp. 97–108. SIAM (2018)

  22. Freuder, E.C.: A sufficient condition for backtrack-free search. Journal of the ACM (JACM) 29(1), 24–32 (1982)

    Article  MathSciNet  Google Scholar 

  23. Fulkerson, D., Ford, L.: Flows in Networks. Princeton University Press, Princeton (1962)

    MATH  Google Scholar 

  24. Gabow, H.N.: Data structures for weighted matching and nearest common ancestors with linking. In: Proceedings of the first annual ACM-SIAM symposium on Discrete algorithms, pp. 434–443 (1990)

  25. Globerson, A., Jaakkola, T.: Fixing max-product: Convergent message passing algorithms for MAP LP-relaxations. In: Neural Information Processing Systems, pp. 553–560 (2008)

  26. Goldberg, A.V., Kennedy, R.: An efficient cost scaling algorithm for the assignment problem. Math. Program. 71(2), 153–177 (1995)

    Article  MathSciNet  Google Scholar 

  27. Hochbaum, D.S.: Solving integer programs over monotone inequalities in three variables: A framework for half integrality and good approximations. Eur. J. Oper. Res. 140(2), 291–321 (2002)

    Article  MathSciNet  Google Scholar 

  28. Hochbaum, D.S., Pathria, A.: Forest harvesting and minimum cuts: a new approach to handling spatial constraints. For. Sci. 43(4), 544–554 (1997)

    Google Scholar 

  29. Hochbaum, D.S., Pathria, A.: Approximating a generalization of MAX 2SAT and MIN 2SAT. Discret. Appl. Math. 107(1-3), 41–59 (2000)

    Article  MathSciNet  Google Scholar 

  30. Hooker, J.: Logic-based methods for optimization: combining optimization and constraint satisfaction. Wiley series in discrete mathematics and optimization. Wiley (2000)

  31. Kappes, J.H., Andres, B., Hamprecht, F.A., Schnörr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B.X., Kröger, T., Lellmann, J., Komodakis, N., Savchynskyy, B., Rother, C.: A comparative study of modern inference techniques for structured discrete energy minimization problems. Intl. J. Comput. Vis. 115(2), 155–184 (2015)

    Article  MathSciNet  Google Scholar 

  32. Kohli, R., Krishnamurti, R., Mirchandani, P.: The minimum satisfiability problem. SIAM J. Discret. Math. 7(2), 275–283 (1994)

    Article  MathSciNet  Google Scholar 

  33. Kolmogorov, V.: Convergent tree-reweighted message passing for energy minimization. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1568–1583 (2006)

    Article  Google Scholar 

  34. Kolmogorov, V.: A new look at reweighted message passing. IEEE Trans. Pattern Anal. Mach. Intell. 37(5) (2015)

  35. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions via graph cuts. Tech. rep., Cornell University (2001)

  36. Kovalevsky, V.A., Koval, V.K.: A diffusion algorithm for decreasing the energy of the max-sum labeling problem. Glushkov Institute of Cybernetics, Kiev, USSR. Unpublished (approx.1975)

  37. Kuhn, H.W.: The Hungarian method for the assignment problem. Naval research logistics quarterly 2(1-2), 83–97 (1955)

    Article  MathSciNet  Google Scholar 

  38. Lempitsky, V., Boykov, Y.: Global optimization for shape fitting. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

  39. Lempitsky, V., Boykov, Y., Ivanov, D.: Oriented visibility for multiview reconstruction. In: European Conference on Computer Vision, pp. 226–238. Springer (2006)

  40. Li, C.M., Manya, F.: Maxsat, hard and soft constraints. Handbook of satisfiability 185, 613–631 (2009)

    Google Scholar 

  41. Matoušek, J., Gärtner, B.: Understanding and using linear programming (universitext). Springer (2006)

  42. Papadimitriou, C.H., Steiglitz, K.: Combinatorial optimization: algorithms and complexity. Courier Corporation (1998)

  43. Průša, D., Werner, T.: LP relaxation of the Potts labeling problem is as hard as any linear program. IEEE Trans. Pattern Anal. Mach. Intell. 39(7), 1469–1475 (2017)

    Article  Google Scholar 

  44. Průša, D., Werner, T.: LP relaxations of some NP-hard problems are as hard as any LP. In: ACM-SIAM Symp. on Discrete Algorithm (SODA), pp. 1372–1382. SIAM (2017)

  45. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Intl. J. Comput. Vis. 47(1-3), 7–42 (2002)

    Article  Google Scholar 

  46. Schlesinger, M.I., Antoniuk, K.: Diffusion algorithms and structural recognition optimization problems. Cybern. Syst. Anal. 47, 175–192 (2011)

    Article  MathSciNet  Google Scholar 

  47. Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)

    Article  Google Scholar 

  48. Tourani, S., Shekhovtsov, A., Rother, C., Savchynskyy, B.: MPLP++: Fast, parallel dual block-coordinate ascent for dense graphical models. In: Proceedings of the European Conference on Computer Vision, pp. 251–267 (2018)

  49. Tourani, S., Shekhovtsov, A., Rother, C., Savchynskyy, B.: Taxonomy of dual block-coordinate ascent methods for discrete energy minimization. arXiv.org (2020)

  50. Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109(3), 475–494 (2001)

    Article  MathSciNet  Google Scholar 

  51. Vazirani, V.V.: Approximation Algorithms. Springer, New York (2001)

    MATH  Google Scholar 

  52. Werner, T.: A linear programming approach to max-sum problem: A review. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1165–1179 (2007)

    Article  Google Scholar 

  53. Werner, T., Průša, D., Dlask, T.: Relative interior rule in block-coordinate descent. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7559–7567 (2020)

  54. Werner, T., Průša, D.: Relative interior rule in block-coordinate minimization. arXiv.org (2019)

  55. Wright, S.J.: Coordinate descent algorithms. Math. Program. 151 (1), 3–34 (2015)

    Article  MathSciNet  Google Scholar 

  56. Xu, K., Li, W.: Many hard examples in exact phase transitions with application to generating hard satisfiable instances. arXiv.org. Instances available at http://sites.nlsde.buaa.edu.cn/kexu/benchmarks/max-sat-benchmarks.htm (2003)

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Czech Technical University in Prague, Karlovo náměstí 13, 120 00 Nové Město, Prague, Czech Republic

    Tomáš Dlask & Tomáš Werner

Authors
  1. Tomáš Dlask
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomáš Werner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tomáš Dlask.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work has been supported by the Czech Science Foundation (grant 19-09967S), the OP VVV project CZ.02.1.01/0.0/0.0/16_019/0000765, and the Grant Agency of the Czech Technical University in Prague (grant SGS19/170/OHK3/3T/13).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dlask, T., Werner, T. Classes of linear programs solvable by coordinate-wise minimization. Ann Math Artif Intell 90, 777–807 (2022). https://doi.org/10.1007/s10472-021-09731-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-021-09731-9

Keywords

Mathematics Subject Classification 2010

Search

Navigation