Abstract
Recent results in compressed sensing show that, under certain conditions, the sparsest solution to an underdetermined set of linear equations can be recovered by solving a linear program. These results either rely on computing sparse eigenvalues of the design matrix or on properties of its nullspace. So far, no tractable algorithm is known to test these conditions and most current results rely on asymptotic properties of random matrices. Given a matrix A, we use semidefinite relaxation techniques to test the nullspace property on A and show on some numerical examples that these relaxation bounds can prove perfect recovery of sparse solutions with relatively high cardinality.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Affentranger F., Schneider R.: Random projections of regular simplices. Discrete Comput. Geom. 7(1), 219–226 (1992)
Barvinok A.: Integration and optimization of multivariate polynomials by restriction onto a random subspace. Foundations Comput. Math. 7(2), 229–244 (2007)
Boyd, S., El Ghaoui, L., Feron, E., Balakrishnan, V.: Linear matrix inequalities in system and control theory. SIAM (1994)
Candès E., Tao T.: Near-optimal signal recovery from random projections: universal encoding strategies?. IEEE Trans. Inform. Theory 52(12), 5406–5425 (2006)
Candès E.J., Tao T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215 (2005)
Cohen A., Dahmen W., DeVore R.: Compressed sensing and best k-term approximation. J. AMS 22(1), 211–231 (2009)
d’Aspremont A., El Ghaoui L., Jordan M., Lanckriet G.R.G.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (2007)
d’Aspremont A., Bach F., El Ghaoui L.: Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9, 1269–1294 (2008)
Donoho D., Huo X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inform. Theory 47(7), 2845–2862 (2001)
Donoho D., Tanner J.: Counting the faces of randomly-projected hypercubes and orthants, with applications. Arxiv Preprint arXiv, 08073590 (2008)
Donoho D.L., Tanner J.: Sparse nonnegative solutions of underdetermined linear equations by linear programming. Proc. Natl. Acad. Sci. 102(27), 9446–9451 (2005)
Goemans M., Williamson D.: Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J ACM 42, 1115–1145 (1995)
Ibragimov, I., Sudakov, V., Tsirelson, B.: Norms of Gaussian sample functions. In: Proceedings of the third Japan USSR symposium on probability theory, lecture notes in math, vol. 550, pp. 20–41 (1976)
Juditsky, A., Nemirovski, A.: On verifiable sufficient conditions for sparse signal recovery via ℓ 1 minimization. ArXiv 08092650 (2008)
Lee, K., Bresler, Y.: Computing performance guarantees for compressed sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, pp 5129–5132 (2008)
Massart, P.: Concentration inequalities and model selection. Ecole d’Eté de Probabilités de Saint-Flour XXXIII (2007)
Moler C., Van Loan C.: Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Rev. 45(1), 3–49 (2003)
Nesterov Y.: Introductory Lectures on Convex Optimization. Springer, Heidelberg (2003)
Nesterov Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)
Nesterov Y.: Smoothing technique and its applications in semidefinite optimization. Math. Program. 110(2), 245–259 (2007)
Sturm J.: Using SEDUMI 1.0x, a MATLAB toolbox for optimization over symmetric cones. Optim. Method. Sofw. 11, 625–653 (1999)
Toh K.C., Todd M.J., Tutuncu R.H.: SDPT3—a MATLAB software package for semidefinite programming. Optim. Method. Sofw. 11, 545–581 (1999)
Vershik A., Sporyshev P.: Asymptotic behavior of the number of faces of random polyhedra and the neighborliness problem. Selecta Math Soviet 11(2), 181–201 (1992)
Zhang Y.: A simple proof for recoverability of ℓ 1-minimization: Go over or under. Rice University CAAM Technical report TR05-09 (2005)
Acknowledgments
The authors are grateful to Arkadi Nemirovski (who suggested in particular the columnwise redundant constraints in (13) and the performance bounds) and Anatoli Juditsky for very helpful comments and suggestions. Would like to thank Ingrid Daubechies for first attracting our attention to the nullspace property. We are also grateful to two anonymous referees for numerous comments and suggestions. We thank Jared Tanner for forwarding us his numerical results. Finally, we acknowledge support from NSF grants DMS-0625352, SES-0835550 (CDI), CMMI-0844795 (CAREER) and CMMI-0968842, a Peek junior faculty fellowship and a Howard B. Wentz Jr. junior faculty award.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
d’Aspremont, A., El Ghaoui, L. Testing the nullspace property using semidefinite programming. Math. Program. 127, 123–144 (2011). https://doi.org/10.1007/s10107-010-0416-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-010-0416-0