Abstract
We consider a class of optimization problems for sparse signal reconstruction which arise in the field of compressed sensing (CS). A plethora of approaches and solvers exist for such problems, for example GPSR, FPC_AS, SPGL1, NestA, \(\mathbf{\ell _1\_\ell _s}\), PDCO to mention a few. CS applications lead to very well conditioned optimization problems and therefore can be solved easily by simple first-order methods. Interior point methods (IPMs) rely on the Newton method hence they use the second-order information. They have numerous advantageous features and one clear drawback: being the second-order approach they need to solve linear equations and this operation has (in the general dense case) an \({\mathcal {O}}(n^3)\) computational complexity. Attempts have been made to specialize IPMs to sparse reconstruction problems and they have led to interesting developments implemented in \(\mathbf{\ell _1\_\ell _s}\) and PDCO softwares. We go a few steps further. First, we use the matrix-free IPM, an approach which redesigns IPM to avoid the need to explicitly formulate (and store) the Newton equation systems. Secondly, we exploit the special features of the signal processing matrices within the matrix-free IPM. Two such features are of particular interest: an excellent conditioning of these matrices and the ability to perform inexpensive (low complexity) matrix–vector multiplications with them. Computational experience with large scale one-dimensional signals confirms that the new approach is efficient and offers an attractive alternative to other state-of-the-art solvers.
Similar content being viewed by others
References
Ahmed, N., Rao, K.R.: Orthogonal transforms for digital signal processing. Springer, Berlin (1975)
Baraniuk, R., Davenport, M., DeVore, R., Wakin, M.: A simple proof of the restricted isometry property for random matrices. Constr. Approx. 28(3), 253–263 (2008)
Becker, S., Bobin, J., Candés, E.J.: Nesta: a fast and accurate first-order method for sparse recovery. SIAM J. Imaging Sci. 4(1), 1–39 (2011)
Van Den Berg, E., Friedlander, M.P.: Probing the Pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)
Van Den Berg, E., Friedlander, M.P., Hennenfent, G., Herrman, F.J., Saab, R., Yılmaz, Ö.: Sparco: a testing framework for sparse reconstruction. ACM Trans. Math. Softw. 35(4), 1–16 (2009)
Blanchard, J.D., Cartis, C., Tanner, J.: Decay properties of restricted isometry constants. Comput. Optim. Appl. 16(7), 572–575 (2009)
Blanchard, J.D., Cartis, C., Tanner, J.: Compressed sensing: how sharp is the restricted isometry property? SIAM Rev. 53(1), 105–125 (2011)
Candés, E., Demanet, L., Donoho, D., Ying, L.: Curvelab. http://www.curvelet.org/software.html
Candés, E.J.: Compressive sampling. In: Proceedings of the International Congress of Mathematicians (2006)
Candés, E.J., Romberg, J.: Practical signal recovery from random projections. In: Proceedings of the SPIE Conference on Wavelet Applications in Signal and Image Processing XI, 5914 (2004)
Candés, E.J., Romberg, J.: \(\ell _1\)-magic. Technical Report, Caltech. http://users.ece.gatech.edu/justin/l1magic/ (2007)
Candés, E.J., Romberg, J.: Sparsity and incoherence in compressive sampling. Inverse Probl. 23(3), 969–985 (2007)
Candés, E.J., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59(8), 1207–1223 (2006)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)
Coifman, R., Geshwind, F., Meyer, Y.: Noiselets. Appl. Comput. Harmon. Anal. 10(1), 27–44 (2001)
Donoho, D.L., Huo, X.: Uncertainty principles and ideal atomic decomposition. IEEE Trans. Inf. Theory 47(7), 2845–2862 (2001)
Donoho, D.L., Johnstone, I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)
Donoho, D.L., Tanner, J.: Precise undersampling theorems. Proc. IEEE 98(6), 913–924 (2010)
Dossal, C., Mallat, S.: Sparse spike deconvolution with minimum scale. Proc. Signal Process. Adapt. Sparse Struct. Represent. 81(3), 123–126 (1994)
Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)
Foucart, S.: A note on guaranteed sparse recovery via 1-minimization. Appl. Comput. Harmon. Anal. 29(1), 97–103 (2010)
Frigo, M., Johnston, S.G.: The design and implementation of FFTW3. Proc. IEEE 93(2), 216–231 (2005)
Gondzio, J.: Multiple centrality corrections in a primal–dual method for linear programming. Comput. Optim. Appl. 6, 137–156 (1996)
Gondzio, J.: Interior point methods 25 years later. Eur. J. Oper. Res. 218(3), 587–601 (2012)
Gondzio, J.: Matrix-free interior point method. Comput. Optim. Appl. 51(2), 457–480 (2012)
Gondzio, J.: Convergence analysis of an inexact feasible interior point method for convex quadratic programming. SIAM J. Optim. 23(3), 1510–1527 (2013)
Hennenfent, G., Herrmann, F.J.: Random sampling: new insights into the reconstruction of coarsely-sampled wavefields. In: SEG International Exposition and 77th Annual Meeting (2007)
Kelley, C.T.: Iterative methods for linear and nonlinear equations, volume 16 of Frontiers in Applied Mathematics. SIAM, Philadelphia (1995)
Kim, S.-J., Koh, K., Lustig, M., Boyd, S., Gorinevsky, D.: An interior-point method for large-scale \(\ell _1\)-regularized least squares. IEEE J. Sel. Top. Signal Process. 1(4), 606–617 (2007)
Kojima, M., Megiddo, N., Mizuno, S.: A primal-dual infeasible-interior-point algorithm for linear programming. Math. Progr. 61, 263–280 (1993)
Lu, Z., Monteiro, R.D.S., O’Neal, J.W.: An iterative solver-based infeasible primal–dual path-following algorithm for convex quadratic programming. SIAM J. Optim. 17, 287–310 (2006)
Miller, A.J.: Subset selection in regression. Chapmain & Hall/CRC, London (2002)
Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)
Rockafellar, R.T.: Convex analysis, Princeton landmarks in mathematics and physics. Princeton University Press, USA (1970)
Rudelson, M., Vershynin, R.: On sparse reconstruction from Fourier and Gaussian measurements. Commun. Pure Appl. Math. 61(8), 1025–1045 (2008)
Saunders, M., Kim, B.: PDCO: Primal-dual interior method for convex objectives. Technical Report, Stanford University. http://www.stanford.edu/group/SOL/software/pdco.html (2002)
Takhar, D., Laska, J.N., Wakin, M., Duarte, M., Baron, D., Sarvotham, S., Kelly, K. K., Baraniuk, R.G.: A new camera architecture based on optical-domain compression. In: Proceedings of the IS &T/SPIE Symposium on Electronic Imaging: Computational, Imaging, 6065 (2006)
Thomson, A.: Compressive single-pixel imaging. In: Proceedings of the 2nd IMA Conference on Mathematics in Defence, Defence Academy, Shrivenham, UK (2011)
Wen, Z., Yin, W., Goldfarb, D., Zhang, Y.: A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization and continuation. SIAM J. Sci. Comput. 32(4), 1809–1831 (2010)
Wright, S.J.: Primal–dual interior-point methods. SIAM, Philadelphia (1997)
Author information
Authors and Affiliations
Corresponding author
Additional information
J. Gondzio is supported by EPSRC Grant EP/I017127/1.
Rights and permissions
About this article
Cite this article
Fountoulakis, K., Gondzio, J. & Zhlobich, P. Matrix-free interior point method for compressed sensing problems. Math. Prog. Comp. 6, 1–31 (2014). https://doi.org/10.1007/s12532-013-0063-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12532-013-0063-6
Keywords
- Matrix-free interior point
- Preconditioned conjugate gradient
- Compressed sensing
- Compressive sampling
- \(\ell _1\)-regularization