Skip to main content
SpringerLink
Log in

Structured Sparsity Promoting Functions

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

Motivated by the minimax concave penalty-based variable selection in high-dimensional linear regression, we introduce a simple scheme to construct structured sparsity promoting functions from convex sparsity promoting functions and their Moreau envelopes. Properties of these functions are developed by leveraging their structure. In particular, we provide sparsity guarantees for the general family of functions. We further study the behavior of the proximity operators of several special functions, including indicator functions of closed and convex sets, piecewise quadratic functions, and linear combinations of the two. To demonstrate these properties, several concrete examples are presented and existing instances are featured as special cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Candes, E., Tao, T.: Near optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press, New York, USA (1999)

    MATH  Google Scholar 

  3. Suter, B.W.: Multirate and Wavelet Signal Processing. Academic Press, San Diego, USA (1997)

    MATH  Google Scholar 

  4. Tibshirani, R.: Regression shrinkage and selection via the LASSO. J. Roy. Stat. Soc. B 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  5. Frank, I.E., Friedman, J.H.: A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109–148 (1993)

    Article  MATH  Google Scholar 

  6. Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  7. Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bolte, J., Daniilidis, A., Ley, O., Mazet, L.: Characterizations of łojasiewicz inequalities: subgradient flows, talweg, convexity. Trans. Am. Math. Soc. 362, 3319–3363 (2010)

    Article  MATH  Google Scholar 

  9. Cannarsa, P., Sinestrari, C.: Semiconcave Functions, Hamilton–Jacobi Equations, and Optimal Control, Progress in Nonlinear Differential Equations and Their Applications, vol. 58. Birkhauser, Boston, USA (2004)

    MATH  Google Scholar 

  10. Tao, P., Le Thi, H.: Convex analysis approach to dc programming: theory, algorithms and applications. Acta Mathematica Vietnamica 22(1), 289–355 (1997)

    MathSciNet  MATH  Google Scholar 

  11. Soubies, E., Blanc-Feraud, L., Aubert, G.: A unified view of exact continuous penalties for \(\ell _2\)-\(\ell _0\) minimization. SIAM J. Optim. 27(3), 2034–2060 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  12. Moreau, J.J.: Proximité et dualité dans un espace hilbertien. Bull. Soc. Math. France 93, 273–299 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  13. Attouch, H., Bolte, J., Svaiter, B.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. Ser. A 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Bauschke, H.L., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. AMS Books in Mathematics. Springer, New York (2011)

    Book  MATH  Google Scholar 

  15. Combettes, P., Wajs, V.: Signal recovery by proximal forward–backward splitting. Multiscale Model. Simul. SIAM Interdiscip. J. 4, 1168–1200 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  16. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chen, S., Donoho, D., Saunders, M.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20, 33–61 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  18. Selesnick, I.: Total variation denoising via the moreau envelope. IEEE Signal Process. Lett. 24(2), 216–220 (2017)

    Article  MathSciNet  Google Scholar 

  19. Hiriart-Urruty, J.B., Lemarechal, C.: Convex Analysis and Minimization Algorithms I, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 305. Springer-Verlag, Berlin (1993)

    Google Scholar 

  20. Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Prob. 27, 045,009 (2011). (30pp)

    Article  MathSciNet  MATH  Google Scholar 

  21. Donoho, D., Johnstone, I.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425–455 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  22. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)

    MATH  Google Scholar 

  23. Planiden, C., Wang, X.: Epi-convergence: the Moreau envelope and generalized linear-quadratic functions. J. Optim. Theory Appl. 177(1), 21–63 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  24. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. Roy. Stat. Soc. B 67, 301–320 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  25. Donoho, D.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  26. Huber, P.: Robust Statistics, 2nd edn. John Wiley & Sons Inc., Hoboken, New Jersey (2009)

    Book  MATH  Google Scholar 

  27. Gao, H.Y., Bruce, A.G.: WaveShrink with firm shrinkage. Stat. Sin. 7, 855–874 (1997)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Lixin Shen is partially supported by the US National Science Foundation under Grant DMS-1522332.

Author information

Authors and Affiliations

  1. Syracuse University, Syracuse, NY, 13244, USA

    Lixin Shen & Erin E. Tripp

  2. Air Force Research Laboratory, Rome, NY, 13441, USA

    Bruce W. Suter

Authors
  1. Lixin Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce W. Suter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erin E. Tripp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lixin Shen.

Additional information

Communicated by Lionel Thibault.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Air Force Research Laboratory (AFRL).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, L., Suter, B.W. & Tripp, E.E. Structured Sparsity Promoting Functions. J Optim Theory Appl 183, 386–421 (2019). https://doi.org/10.1007/s10957-019-01565-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-019-01565-0

Keywords

Mathematics Subject Classification

Search

Navigation