Skip to main content
SpringerLink
Log in

Computing the spark: mixed-integer programming for the (vector) matroid girth problem

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We investigate the NP-hard problem of computing the spark of a matrix (i.e., the smallest number of linearly dependent columns), a key parameter in compressed sensing and sparse signal recovery. To that end, we identify polynomially solvable special cases, gather upper and lower bounding procedures, and propose several exact (mixed-)integer programming models and linear programming heuristics. In particular, we develop a branch and cut scheme to determine the girth of a matroid, focussing on the vector matroid case, for which the girth is precisely the spark of the representation matrix. Extensive numerical experiments demonstrate the effectiveness of our specialized algorithms compared to general-purpose black-box solvers applied to several mixed-integer programming models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We do not include \(\sigma \) as defined in Proposition 5.2 to omit further nodes, as they will be pruned during propagation anyway.

  2. Namely, randrounding, simplerounding, rounding, feaspump, locks, rens, rins, crossover and oneopt.

  3. http://www.graphics.rwth-aachen.de/person/265/.

References

  1. Alexeev, B., Cahill, J., Mixon, D.G.: Full spark frames. J. Fourier Anal. Appl. 18(6), 1167–1194 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amaldi, E., Kann, V.: The complexity and approximability of finding maximum feasible subsystems of linear relations. Theor. Comput. Sci. 147, 181–210 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  3. Amaldi, E., Kann, V.: On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theor. Comput. Sci. 209(1–2), 237–260 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  4. Arellano, J.D.: Algorithms to find the girth and cogirth of a linear matroid. Ph.D. thesis, Rice University (2014)

  5. Arellano, J.D., Hicks, I.V.: Degree of redundancy of linear systems using implicit set covering. IEEE Trans. Autom. Sci. Eng. 11(1), 274–279 (2014)

    Article  Google Scholar 

  6. Arora, S., Babai, L., Stern, J., Sweedyk, Z.: The hardness of approximate optima in lattices, codes, and systems of linear equations. J. Comput. Syst. Sci. 54(2), 317–331 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  7. Aveila, P., Boccia, M., Vasilyev, I.: Computational experience with general cutting planes for the set covering problem. Oper. Res. Lett. 37(1), 16–20 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  8. Aykanat, C., Pinar, A., Çatalyürek, U.: Permuting sparse rectangular matrices into block-diagonal form. SIAM J. Sci. Comput. 25(6), 1860–1879 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  9. Balas, E.: Cutting planes from conditional bounds: a new approach to set covering. Math. Prog. 12, 19–36 (1980)

    MathSciNet  MATH  Google Scholar 

  10. Balas, E., Ho, A.: Set covering algorithms using cutting planes, heuristics, and subgradient optimization: a computational study. Math. Prog. 12, 37–60 (1980)

    MathSciNet  MATH  Google Scholar 

  11. Balas, E., Jeroslow, R.: Canonical cuts on the unit hypercube. SIAM J. Appl. Math. 23(1), 61–69 (1972)

    Article  MathSciNet  MATH  Google Scholar 

  12. Balas, E., Ng, S.M.: On the set covering polytope: I. All the facets with coefficients in 0, 1, 2. Math. Program. 43(1–3), 57–69 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  13. Bausal, M., Kianfar, K., Ding, Y., Moreno-Centeno, E.: Hybridization of bound-and-decompose and mixed integer feasibility checking to measure redundancy in structured linear systems. IEEE Trans. Autom. Sci. Eng. 10(4), 1151–1157 (2013)

    Article  Google Scholar 

  14. Beasley, J.E., Jornsten, K.: Enhancing an algorithm for set covering problems. Eur. J. Oper. Res. 58(2), 293–300 (1992)

    Article  MATH  Google Scholar 

  15. Berry, M.W., Heath, M.T., Kaneka, I., Lawo, M., Plemmons, R.J., Ward, R.C.: An algorithm to compute a sparse basis of the null space. Numer. Math. 47(4), 483–504 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  16. Bixby, R.E., Saltzman, M.J.: Recovering an optimal LP basis from an interior point solution. Oper. Res. Lett. 15(4), 169–178 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  17. Borndörfer, R.: Aspects of set packing, partitioning, and covering. Doctoral dissertation. TU Berlin, Germany (1998)

  18. Cameron, P.J.: Combinatorics: topics, techniques. Algorithms. Cambridge University Press, Cambridge (1994)

    Book  MATH  Google Scholar 

  19. Candès, E., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  20. Caprara, A., Toth, P., Fischetti, M.: Algorithms for the set covering problem. Ann. Oper. Res. 98, 353–371 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chang, S.F., McCormick, S.T.: A hierarchical algorithm for making sparse matrices sparser. Math. Program. 56(1–3), 1–30 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  23. Chistov, A., Fournier, H., Gurvits, L., Koiran, P.: Vandermonde matrices, NP-completeness, and transversal subspaces. Found. Comput. Math. 3(4), 421–427 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  24. Cho, J.J., Chen, Y., Ding, Y.: On the (co)girth of a connected matroid. Discrete Appl. Math. 155(18), 2456–2470 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  25. Cho, M., Mishra, K.V., Xu, W.: Computable performance guarantees for compressed sensing matrices. EURASIP J. Adv. Signal Process. 2018, 16 (2018)

    Article  Google Scholar 

  26. Coleman, T.F., Pothen, A.: The sparse null space basis problem. Technical Report TR 84-598, Cornell University, Ithaca, NY, USA (1984)

  27. Damaschke, P., Eğecioğlu, O., Molokov, L.: Fixed-Parameter Tractability of Error Correction in Graphical Linear Systems. In: Ghosh, S.K., Tokuyama, T. (eds.) Proceedings of the WALCOM. 2013, LNCS, vol. 7748, pp. 245–256. Springer (2013)

  28. d’Aspremont, A., Bach, F., Ghaoui, L.E.: Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9, 1269–1294 (2008)

    MathSciNet  MATH  Google Scholar 

  29. d’Aspremont, A., Ghaoui, L.E.: Testing the nullspace property using semidefinite programming. Math. Program. 127(1), 123–144 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  30. d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.G.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  31. DeVore, R.A.: Deterministic constructions of compressed sensing matrices. J. Complex. 23(4–6), 918–925 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  32. Dimakis, A.G., Smarandache, R., Vontobel, P.O.: LDPC codes for compressed sensing. IEEE Trans. Inf. Theory 58(5), 3093–3114 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  33. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  34. Donoho, D.L., Elad, M.: Optimally sparse representation in general (non-orthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100(5), 2197–2202 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  35. Egner, S., Minkwitz, T.: Sparsification of rectangular matrices. J. Symb. Comput. 26(2), 135–149 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  36. Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, Heidelberg (2010)

    Book  MATH  Google Scholar 

  37. Ferris, M.C., Pataki, G., Schmieta, S.: Solving the seymour problem. Optima 66, 2–6 (2001)

    Google Scholar 

  38. Fischer, T.: Konstruktion von dünn besetzten Sensing-Matrizen. Diploma Thesis, TU Darmstadt, Germany (2012) (in German)

  39. Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numeric Harmonic Analysis. Birkhäuser, Basel (2013)

    MATH  Google Scholar 

  40. Gally, T., Pfetsch, M.E.: Computing Restricted Isometry Constants via Mixed-Integer Semidefinite Programming. Optimization Online E-print (2016). http://www.optimization-online.org/DB_HTML/2016/04/5395.html. Accessed 20 Apr 2018

  41. Garey, M.R., Johnson, D.S.: Computers and Intractability. A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, New York (1979)

    MATH  Google Scholar 

  42. Gharibi, W.: An improved lower bound of the spark with application. Int. J. Distrib. Parallel Syst. 3(5), 1–8 (2012)

    Article  Google Scholar 

  43. Gilbert, J.R., Heath, M.T.: Computing a sparse basis for the null space. SIAM J. Algebraic Discrete Methods 8(3), 446–459 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  44. Gottlieb, L.A., Neylon, T.: Matrix sparsification and the sparse null space problem. Algorithmica 76(2), 426–444 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  45. Gribonval, R., Nielsen, M.: Sparse representations in unions of bases. IEEE Trans. Inf. Theory 49(2), 3320–3325 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  46. Grossman, J.W., Kulkarni, D.M., Schochetman, I.E.: Algebraic graph theory without orientation. Linear Algebra Appl. 212–213, 289–307 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  47. Grossman, J.W., Kulkarni, D.M., Schochetman, I.E.: On the minors of an incidence matrix and its Smith normal form. Linear Algebra Appl. 218, 213–224 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  48. Grötschel, M., Lovász, L., Schrijver, A.: Geometric Algorithms and Combinatorial Optimization, Algorithms and Combinatorics, vol. 2, 2nd edn. Springer, Berlin (1993)

    Book  MATH  Google Scholar 

  49. Helmling, M., Ruzika, S., Tanatmis, A.: Mathematical programming decoding of binary linear codes: theory and algorithms. IEEE Trans. Inf. Theory 58(7), 4753–4769 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  50. Helmling, M., Scholl, S., Gensheimer, F., Dietz, T., Kraft, K., Ruzika, S., Wehn, N.: Database of channel codes and ML simulation results (2017). https://www.uni-kl.de/channel-codes/channelcodes. Accessed 20 Apr 2018

  51. Hoffman, A.J., McCormick, S.T.: A fast algorithm that makes matrices optimally sparse. In: Pulleyblank, W.R. (ed.) Progress in Combinatorial Optimization, pp. 185–196. Academic Press, Amsterdam (1984)

    Chapter  Google Scholar 

  52. Itai, A., Rodeh, M.: Finding a minimum circuit in a graph. SIAM J. Comput. 7(4), 413–423 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  53. Iwen, M.A.: Simple deterministically constructible RIP matrices with sublinear Fourier sampling requirements. Proc. CISS 2009, 870–875 (2009)

    Google Scholar 

  54. Jensen, P.M., Korte, B.: Complexity of matroid property algorithms. SIAM J. Comput. 11(1), 184–190 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  55. Jokar, S., Pfetsch, M.E.: Exact and approximate sparse solutions of underdetermined linear equations. SIAM J. Sci. Comput. 31(1), 23–44 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  56. Juditsky, A., Nemirovski, A.: On verifiable sufficient conditions for sparse signal recovery via \(\ell 1\) minimization. Math. Program. 127(1), 57–88 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  57. Keha, A.B., Duman, T.: Minimum distance computation of LDPC codes using a branch and cut algorithm. IEEE Trans. Commun. 58(4), 1072–1079 (2010)

    Article  Google Scholar 

  58. Khachiyan, L., Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: On the complexity of some enumeration problems for matroids. SIAM J. Discrete Math. 19(4), 966–984 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  59. Kianfar, K., Pourhabib, A., Ding, Y.: An integer programming approach for analyzing the measurement redundancy in structured linear systems. IEEE Trans. Autom. Sci. Eng. 8(2), 447–450 (2011)

    Article  Google Scholar 

  60. King, E.: Algebraic and geometric spread in finite frames. In: Proceedings of the SPIE 9597, Wavelets and Sparsity XVI, Article no. 95970B (2015)

  61. Koiran, P., Zouzias, A.: On the Certification of the Restricted Isometry Property. arXiv:1103.4984 [cs.CC] (2011). Accessed 20 Apr 2018

  62. Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  63. Kruskal, J.B.: Three-way arrays: rank and uniqueness of trilinear decompositions. Linear Algebra Appl. 18(2), 95–138 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  64. Lange, J.H., Pfetsch, M.E., Seib, B.M., Tillmann, A.M.: Sparse Recovery with Integrality Constraints. arXiv:1608.08678 [cs.IT] (2016). Accessed 20 Apr 2018

  65. Lisoněk, P., Trummer, L.: Algorithms for the minimum weight of linear codes. Adv. Math. Commun. 10(1), 195–207 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  66. Liu, X.J., Xia, S.T.: Constructions of quasi-cyclic measurement matrices based on array codes. Proc. ISIT 2013, 479–483 (2013)

    Google Scholar 

  67. Maculan, N., Macambira, E.M., de Souza, C.C.: Geometrical Cuts for 0–1 Integer Programming. Technical Report IC-02-006, Instituto de Computacao, Universidade Estadual de Campinas (2002)

  68. Maher, S.J., Fischer, T., Gally, T., Gamrath, G., Gleixner, A., Gottwald, R.L., Hendel, G., Koch, T., Lübbecke, M.E., Miltenberger, M., Müller, B., Pfetsch, M.E., Puchert, C., Rehfeldt, D., Schenker, S., Schwarz, R., Serrano, F., Shinano, Y., Weninger, D., Witt, J.T., Witzig, J.: The SCIP Optimization Suite 4.0. Technical Report 17-12, ZIB (2017)

  69. Manikas, A., Proukakis, C.: Modeling and estimation of ambiguities in linear arrays. IEEE Trans. Signal Process. 46(8), 2166–2179 (1998)

    Article  Google Scholar 

  70. McCormick, S.T.: A Combinatorial Approach to Some Sparse Matrix Problems. Ph.D. thesis, Stanford University (1983)

  71. McCormick, S.T.: Making sparse matrices sparser: computational results. Math. Program. 49(1–3), 91–111 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  72. Minieka, E.: Finding the circuits of a matroid. J. Res. Natl. Bur. Stand. B 80B(3), 337–342 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  73. Mitchell, J.E.: Branch-and-cut algorithms for combinatorial optimization problems. In: Pardalos, P.M., Resende, M.G.C. (eds.) Handbook of Applied Optimization, pp. 65–77. Oxford Univ. Press, Oxford (2002)

    Google Scholar 

  74. Monien, B.: The complexity of determining a shortest cycle of even length. Computing 31(4), 355–369 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  75. Naidu, R.R., Jampana, P., Sastry, C.S.: Deterministic compressed sensing matrices: construction via Euler squares and applications. IEEE Trans. Signal Process. 64(14), 3566–3575 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  76. Oxley, J.G.: Matroid Theory. Oxford Graduate Texts Mathematics, 2nd edn. Oxford Univ. Press, Oxford (2011)

    Google Scholar 

  77. Robinson, G.C., Welsh, D.J.A.: The computational complexity of matroid properties. Math. Proc. Camb. Philos. Soc. 87(29), 29–45 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  78. Schrijver, A.: Combinatorial optimization: polyhedra and efficiency, volume A algorithms and combinatorics, vol. 24. Springer, Berlin (2003)

    MATH  Google Scholar 

  79. Seymour, P.D.: Decomposition of regular matroids. J. Combin. Theory Ser. B 28, 305–359 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  80. Seymour, P.D.: A note on hyperplane generation. J. Combin. Theory Ser. B 61, 88–91 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  81. Tanatmis, A., Ruzika, S., Hamacher, H.W., Punekar, M., Kienle, F., Wehn, N.: Valid inequalities for binary linear codes. Proc. ISIT 2009, 2216–2220 (2009)

    MATH  Google Scholar 

  82. Tillmann, A.M.: Computational Aspects of Compressed Sensing. Doctoral dissertation. TU Darmstadt, Germany (2013)

  83. Tillmann, A.M.: On the computational intractability of exact and approximate dictionary learning. IEEE Signal Process. Lett. 22(1), 45–49 (2015)

    Article  Google Scholar 

  84. Tillmann, A.M., Gribonval, R., Pfetsch, M.E.: Projection onto the cosparse set is NP-hard. Proc. ICASSP 2014, 7148–7152 (2014)

    Google Scholar 

  85. Tillmann, A.M., Pfetsch, M.E.: The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans. Inf. Theory 60(2), 1248–1259 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  86. Truemper, K.: A decomposition theory for matroids. V. Testing of matrix total unimodularity. J. Combin. Theory Ser. B 49, 241–281 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  87. Vardy, A.: The intractability of computing the minimum distance of a code. IEEE Trans. Inf. Theory 43(6), 1757–1766 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  88. Walter, M., Truemper, K.: Implementation of a unimodularity test. Math. Program. Comput. 5(1), 57–73 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  89. Zhang, J., Han, G., Fang, Y.: Deterministic construction of compressed sensing matrices from protograph LDPC codes. IEEE Signal Process. Lett. 22(11), 1960–1964 (2015)

    Article  Google Scholar 

  90. Zhang, X.D.: Matrix Analysis and Applications. Cambridge Univ. Press, Cambridge (2017)

    Book  MATH  Google Scholar 

  91. Zhu, Z., So, A.M.C., Ye, Y.: Fast and near-optimal matrix completion via randomized basis pursuit. In: Ji, L., Poon, Y.S., Yang, L., Yau, S.T. (eds.) Proceedings of the 5th ICCM. AMS/IP Studies in Advanced Mathematics, vol. 51, pp. 859–882. AMS and International Press (2012)

Download references

Acknowledgements

The author would like to thank Marc Pfetsch for inspiring discussions in the early stages of this research effort and his support in getting started working with the SCIP framework, the authors of [38, 75] for kindly providing code for their respective deterministic compressed sensing matrix construction routines, as well as the anonymous reviewer whose comments helped improve the paper.

Author information

Authors and Affiliations

  1. Visual Computing Institute and Chair of Operations Research, RWTH Aachen University, Lehrstuhl für Informatik 8, 52056, Aachen, Germany

    Andreas M. Tillmann

Authors
  1. Andreas M. Tillmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas M. Tillmann.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tillmann, A.M. Computing the spark: mixed-integer programming for the (vector) matroid girth problem. Comput Optim Appl 74, 387–441 (2019). https://doi.org/10.1007/s10589-019-00114-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-019-00114-9

Keywords

Mathematics Subject Classification

Search

Navigation