, Volume 76, Issue 2, pp 426–444 | Cite as

Matrix Sparsification and the Sparse Null Space Problem



We revisit the matrix problems sparse null space and matrix sparsification, and show that they are equivalent. We then proceed to seek algorithms for these problems: we prove the hardness of approximation of these problems, and also give a powerful tool to extend algorithms and heuristics for sparse approximation theory to these problems.


Matrix sparsification Sparse null space Sparse approximation 



We thank Daniel Cohen for finding an error in an earlier version of this paper, Robi Krauthgamer for helpful discussions, and the anonymous reviewers for many helpful suggestions.


  1. 1.
    Amaldi, E., Kann, V.: The complexity and approximability of finding maximum feasible subsystems of linear relations. Theor. Comput. Sci. 147(1–2), 181–210 (1995)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Arora, S., Babai, L., Stern, J., Sweedyk, Z.: The hardness of approximate optima in lattices, codes and linear equations. J. Comput. Syst. Sci. 54(2), 317–331 (1997)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Berman, Piotr., Karpinski, Marek.: Approximating minimum unsatisfiability of linear equations. In: Proceedings of the Thirteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’02, pp. 514–516. Society for Industrial and Applied Mathematics (2002)Google Scholar
  4. 4.
    Berry, M., Heath, M., Kaneko, I., Lawo, M., Plemmons, R., Ward, R.: An algorithm to compute a sparse basis of the null space. Numer. Math. 47(4), 483–504 (1985)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Brualdi, R.A., Friedland, S., Pothen, A.: The sparse basis problem and multilinear algebra. SIAM J. Matrix Anal. Appl. 16(1), 1–20 (1995)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Candès, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Chang, S.Frank, McCormick, S.Thomas: A hierarchical algorithm for making sparse matrices sparser. Math. Program. 56(1–3), 1–30 (1992)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Chartrand, R.: Nonconvex compressed sensing and error correction. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP ’07, volume 3, pages III–889–III–892, (April 2007)Google Scholar
  10. 10.
    Coifman, R., Wickerhauser, M.: Entropy-based algorithms for best basis selection. IEEE Trans. Inf. Theory 38(2), 713–718 (1992)CrossRefMATHGoogle Scholar
  11. 11.
    Coleman, T.F., Pothen, A.: The null space problem I. Complexity. SIAM J. Algebraic Discrete Methods 7(4), 527–537 (1986)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Dinur, Irit, Kindler, Guy, Raz, Ran, Safra, Shmuel: Approximating CVP to within almost-polynomial factors is NP-hard. Combinatorica 23(2), 205–243 (2003)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Donoho, David L.: For most large underdetermined systems of linear equations the minimal \(\ell _{1}\)-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Duff, I.S., Erisman, A.M., Reid, J.K.: Direct Methods for Sparse Matrices. Oxford University Press, Oxford (1986)MATHGoogle Scholar
  15. 15.
    Fuchs, J.J.: On sparse representations in arbitrary redundant bases. IEEE Trans. Inf. Theory 50(6), 1341–1344 (2004)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Gilbert, A.C., Guha, S., Indyk, P., Muthukrishnan, S., Strauss, M.: Near-optimal sparse fourier representations via sampling. In: Proceedings of the Thiry-fourth Annual ACM Symposium on Theory of Computing, STOC ’02, pp. 152–161. ACM (2002)Google Scholar
  17. 17.
    Gilbert, A.C., Muthukrishnan, S., Strauss, M.: Improved time bounds for near-optimal sparse fourier representations. In: Proceedings of SPIE Wavelets XI, vol. 5914, (2005)Google Scholar
  18. 18.
    Gilbert, J.R., Heath, M.T.: Computing a sparse basis for the null space. SIAM J Algebraic Discrete Methods 8(3), 446–459 (1987)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Higham, Nicholas J.: Accuracy and Stability of Numerical Algorithms. In: Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2nd edition, 2002. Chapter 28: A Gallery of Test Matrices (2002)Google Scholar
  20. 20.
    Hoffman, A.J., McCormick, S.T.: A fast algorithm that makes matrices optimally sparse. In: Progress in Combinatorial Optimization. Academic Press (1984)Google Scholar
  21. 21.
    Johnson, D.S., Preparata, F.P.: The densent hemisphere problem. Theor. Comput. Sci. 6(1), 93–107 (1978)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Kann, Viggo: Polynomially bounded minimization problems that are hard to approximate. Nordic J. Comput. 1(3), 317–331 (1994)MathSciNetMATHGoogle Scholar
  23. 23.
    Kavitha, T., Mehlhorn, K., Michail, D., Paluch, K.: A faster algorithm for minimum cycle basis of graphs. In: Diaz, J., Karhumäki, J., Lepistö, A., Sannella, D. (eds.) Automata, Languages and Programming, vol. 3142 of Lecture Notes in Computer Science, pp. 846–857. Springer, Berlin (2004)Google Scholar
  24. 24.
    MacWilliams, F.J., Sloane, N.J.A.: The theory of error-correcting codes. Amsterdam, North-Holland (1983)Google Scholar
  25. 25.
    Matous̆ek, J., Gartner, B.: Understanding and Using Linear Programming. Springer, NewYork (2007)Google Scholar
  26. 26.
    McCormick, S.T.: A combinatorial approach to some sparse matrix problems. Ph.D. thesis, Stanford Univ., Stanford, California, 1983. Technical Report 83-5, Stanford Optimization Lab (1983)Google Scholar
  27. 27.
    Muthukrishnan, S.: Nonuniform sparse approximation with Haar wavelet basis. DIMACS TR:2004-42, (2004)Google Scholar
  28. 28.
    Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Needell, D., Tropp, J.: Cosamp: iterative signal recovery from incomplete and inaccurate samples. Appl. Comput. Harmon. Anal. 26(3), 301–322 (2009)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Neylon, T.: Sparse Solutions for Linear Prediction Problems. Ph.D. thesis, New York University, New York (2006)Google Scholar
  31. 31.
    Pothen, A.: Sparse Null Bases and Marriage Theorems. Ph.D. thesis, Cornell University, Ithica, New York (1984)Google Scholar
  32. 32.
    Schmidt, E.: Zur thoerie der linearen und nichtlinearen integralgleichungen, I. Mathematische Annalen 64(4), 433–476 (1906–1907)Google Scholar
  33. 33.
    Singhal, S.: Amplitude optimization and pitch prediction in multi-pulse coders. IEEE Trans. Acoust Speech Signal Process 37(3), 317–327 (1989)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Smola, Alex J., Schökopf, Bernhard.: Sparse greedy matrix approximation for machine learning. In Proceedings of the Seventeenth International Conference on Machine Learning, ICML ’00, pp. 911–918 (2000)Google Scholar
  35. 35.
    Temlyakov, V.: Nonlinear methods of approximation. Found. Comput. Math. 3(1), 33–107 (2003)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Trefethen, L.N., Bau, D.: Numerical Linear Algebra. SIAM, Philadelphia, PA (1997)Google Scholar
  37. 37.
    Tropp, J.A.: Greed is good: algorithmic results for sparse approximation. IEEE Trans. Inf. Theory 50(10), 2231–2242 (2004)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Tropp, J.A.: Just relax: convex programming methods for subset selection and sparse approximation. IEEE Trans. Inf. Theory 50(3), 1030–1051 (2006)MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Tropp, J.A.: Recovery of short, complex linear combinations via \(\ell _1\) minimization. IEEE Trans. Inf. Theory 51(1), 188–209 (2006)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Wang, X.: A simple proof of Descartes’ rule of signs. Am. Math. Mon. 111(6), 525–526 (2004)CrossRefMATHGoogle Scholar
  41. 41.
    Wright, John, Ma, Yi: Dense error correction via l1-minimization. IEEE Trans. Inf. Theor. 56(7), 3540–3560 (2010)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Zhao, X., Zhang, X., Neylon, T., Shasha, D.: Incremental methods for simple problems in time series: algorithms and experiments. In Ninth International Database Engineering and Applications Symposium (IDEAS 2005), pages 3–14 (2005)Google Scholar
  43. 43.
    Zhou, J., Gilbert, A., Strauss, M., Daubechies, I.: Theoretical and experimental analysis of a randomized algorithm for sparse fourier transform analysis. J. Comput. Phys. 211, 572–595 (2006)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Ariel UniversityArielIsrael
  2. 2.Bynomial, Inc.BerkeleyUSA

Personalised recommendations