Restricted Normal Cones and Sparsity Optimization with Affine Constraints

Abstract

The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is inherently satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    We set \(\operatorname {sgn}(0):=0\).

  2. 2.

    The collection (A i ) iI is said to be nontrivial if I≠∅.

  3. 3.

    When there is no cause for confusion, we shall write column vectors as row vectors for space reasons.

References

  1. 1.

    N. Aronszajn, Theory of reproducing kernels, Trans. Am. Math. Soc. 68, 337–404 (1950).

    Article  MATH  MathSciNet  Google Scholar 

  2. 2.

    H. Attouch, J. Bolte, P. Redont, A. Soubeyran, Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kuryka–Lojasiewicz inequality, Math. Oper. Res. 35, 438–457 (2010).

    Article  MATH  MathSciNet  Google Scholar 

  3. 3.

    H.H. Bauschke, P.L. Combettes, Convex Analysis and Monotone Operator Theory in Hilbert Spaces (Springer, New York, 2011).

    Google Scholar 

  4. 4.

    H.H. Bauschke, S.G. Kruk, Reflection–projection method for convex feasibility problems with an obtuse cone, J. Optim. Theory Appl. 120, 503–531 (2004).

    Article  MATH  MathSciNet  Google Scholar 

  5. 5.

    H.H. Bauschke, D.R. Luke, H.M. Phan, X. Wang, Restricted normal cones and the method of alternating projections: theory, Set-Valued Var. Anal. (2013). doi:10.1007/s1122801302392.

    Google Scholar 

  6. 6.

    H.H. Bauschke, D.R. Luke, H.M. Phan, X. Wang, Restricted normal cones and the method of alternating projections: applications, Set-Valued Var. Anal. (2013). doi:10.1007/s1122801302383.

    Google Scholar 

  7. 7.

    A. Beck, M. Teboulle, A linearly convergent algorithm for solving a class of nonconvex/affine feasibility problems, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, ed. by H.H. Bauschke, R.S. Burachik, P.L. Combettes, V. Elser, D.R. Luke, H. Wolkowicz (Springer, New York, 2011), pp. 33–48.

    Google Scholar 

  8. 8.

    J.M. Borwein, D.R. Luke, Entropic regularization of the 0 function, in Fixed-Point Algorithms for Inverse Problems in Science and Engineering, ed. by H.H. Bauschke, R.S. Burachik, P.L. Combettes, V. Elser, D.R. Luke, H. Wolkowicz (Springer, New York, 2011), pp. 65–92.

    Google Scholar 

  9. 9.

    J.M. Borwein, Q.J. Zhu, Techniques of Variational Analysis (Springer, New York, 2005).

    Google Scholar 

  10. 10.

    A.M. Bruckstein, D.L. Donoho, M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev. 51, 34–81 (2009).

    Article  MATH  MathSciNet  Google Scholar 

  11. 11.

    E. Candes, T. Tao, Near-optimal signal recovery from random projections: universal encoding strategies?, IEEE Trans. Inf. Theory 52, 5406–5425 (2006).

    Article  MathSciNet  Google Scholar 

  12. 12.

    Y. Censor, S.A. Zenios, Parallel Optimization (Oxford University Press, New York, 1997).

    Google Scholar 

  13. 13.

    P.L. Combettes, H.J. Trussell, Method of successive projections for finding a common point of sets in metric spaces, J. Optim. Theory Appl. 67, 487–507 (1990).

    Article  MATH  MathSciNet  Google Scholar 

  14. 14.

    F. Deutsch, Best Approximation in Inner Product Spaces (Springer, New York, 2001).

    Google Scholar 

  15. 15.

    D.L. Donoho, M. Elad, Optimally sparse representation in general (nonorthogonal) dictionaries via 1 minimization, Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003).

    Article  MATH  MathSciNet  Google Scholar 

  16. 16.

    K. Friedrichs, On certain inequalities and characteristic value problems for analytic functions and for functions of two variables, Trans. Am. Math. Soc. 41, 321–364 (1937).

    Article  MathSciNet  Google Scholar 

  17. 17.

    M.J. Lai, J. Wang, An unconstrained q minimization for sparse solution of underdetermined linear systems, SIAM J. Optim. 21, 82–101 (2010).

    Article  MathSciNet  Google Scholar 

  18. 18.

    A.S. Lewis, J. Malick, Alternating projection on manifolds, Math. Oper. Res. 33, 216–234 (2008).

    Article  MATH  MathSciNet  Google Scholar 

  19. 19.

    A.S. Lewis, D.R. Luke, J. Malick, Local linear convergence for alternating and averaged nonconvex projections, Found. Comput. Math. 9, 485–513 (2009).

    Article  MATH  MathSciNet  Google Scholar 

  20. 20.

    D.R. Luke, Local linear convergence and approximate projections onto regularized sets, Nonlinear Anal. 75, 1531–1546 (2012).

    Article  MATH  MathSciNet  Google Scholar 

  21. 21.

    D.R. Luke, Prox-regularity of rank constraint sets and implications for algorithms, J. Math. Imaging Vis. (2012). doi:10.1007/s1085101204063.

    Google Scholar 

  22. 22.

    B.S. Mordukhovich, Variational Analysis and Generalized Differentiation I (Springer, New York, 2006).

    Google Scholar 

  23. 23.

    B.K. Natarajan, Sparse approximate solutions to linear systems, SIAM J. Comput. 24, 227–234 (1995).

    Article  MATH  MathSciNet  Google Scholar 

  24. 24.

    R.T. Rockafellar, Convex Analysis (Princeton University Press, Princeton, 1970).

    Google Scholar 

  25. 25.

    R.T. Rockafellar, R.J.-B. Wets, Variational Analysis (Springer, New York, 2009). Corrected 3rd printing.

    Google Scholar 

Download references

Acknowledgements

Research of H.H. Bauschke was supported in part by the Natural Sciences and Engineering Research Council of Canada and by the Canada Research Chair Program. Research of D.R. Luke was supported in part by the German Research Foundation grant SFB755-A4. Research of H.M. Phan was supported in part by the Pacific Institute for the Mathematical Sciences and by a University of British Columbia research grant. Research of X. Wang was supported in part by the Natural Sciences and Engineering Research Council of Canada.

Author information

Affiliations

Authors

Corresponding author

Correspondence to D. Russell Luke.

Additional information

Communicated by Emmanuel Candés.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bauschke, H.H., Luke, D.R., Phan, H.M. et al. Restricted Normal Cones and Sparsity Optimization with Affine Constraints. Found Comput Math 14, 63–83 (2014). https://doi.org/10.1007/s10208-013-9161-0

Download citation

Keywords

  • Alternating projections
  • Compressed sensing
  • Constraint qualification
  • Friedrichs angle
  • Linear convergence
  • Normal cone
  • Projection operator
  • Restricted normal cone
  • Sparse feasibility
  • Sparsity optimization
  • Superregularity
  • Variational analysis

Mathematics Subject Classification (2010)

  • 49J52
  • 49M20
  • 90C30
  • 15A29
  • 47H09
  • 65K05
  • 65K10
  • 94A08