The problem of convex optimization is studied. Usually in convex optimization the minimization is over a d-dimensional domain. Very often the convergence rate of an optimization algorithm depends on the dimension d. The algorithms studied in this paper utilize dictionaries instead of a canonical basis used in the coordinate descent algorithms. We show how this approach allows us to reduce dimensionality of the problem. Also, we investigate which properties of a dictionary are beneficial for the convergence rate of typical greedy-type algorithms.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
J. M. Borwein and A. S. Lewis, Convex Analysis and Nonlinear Optimization. Theory and Examples, CMS Books in Mathematics/Ouvrages de Mathmatiques de la SMC, 3, Springer (New York, 2006).
R. A. DeVore, Deterministic Constructions of Compressed Sensing Matrices, J. Complexity, 23 (2007), 918–925.
R. A. DeVore and V. N. Temlyakov, Convex optimization on Banach spaces, Found. Comput. Math., published online: February 14, 2015, 1–26; arXiv:1401.0334v1, 1 Jan 2014.
I. Dumer, Covering spheres with spheres, Discrete Comput. Geom., 38 (2007), 665–679.
M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems, IEEE J. Sel. Topics Signal Process., 1 (2007), 586–597.
A. C. Gilbert, S. Muthukrishnan and M. J. Strauss, Approximation of functions over redundant dictionaries using coherence, Proc. of the 14th Annual ACM-SIAM Symposium on Discrete Algorithms (Baltimore, MD, 2003), ACM (New York 2003), 243–252.
B. S. Kashin, On widths of octahedrons, Uspekhi Mat. Nauk, 30 (1975), 251–252.
J. L. Nelson and V. N. Temlyakov, On the size of incoherent systems, J. Approx. Theory, 163 (2011), 1238–1245.
Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer Academic Publishers (Boston, 2004).
C. A. Rogers, Covering a sphere with spheres, Mathematika, 10 (1963), 157–164.
V. N. Temlyakov, Greedy algorithms in Banach spaces, Adv. Comput. Math., 14 (2001), 277–292.
V. N. Temlyakov, Greedy approximation, Cambridge University Press (Cambridge, 2011).
V. N. Temlyakov, Greedy approximation in convex optimization, Constr. Approx., 41 (2015), 269–296; arXiv: 1206.0392v1 [stat.ML] 2 Jun 2012.
V. N. Temlyakov, Greedy expansions in convex optimization, Proc. Steklov Inst. Math., 284 (2014), 244–262; arXiv: 1206.0393v1 [stat.ML] 2 Jun 2012.
V. N. Temlyakov, Chebyshev Greedy Algorithm in convex optimization, IMI Preprint, 2013:08, 1–11; arXiv:1312.1244v1, 4 Dec 2013.
V. N. Temlyakov, Sparse approximation and recovery by greedy algorithms in Banach spaces, Forum Math. Sigma, 2 (2014), e12, pp. 26; IMI Preprint, 2013:09,1–27; arXiv:1303.6811v1, 27 Mar 2013.
V. N. Temlyakov, Incoherent systems and covering in finite-dimensional Banach spaces, Mat. Sb., 205 (2014), 97–116.
J.-L. Verger-Gaugry, Covering a ball with smaller equal balls in Rn, Discrete Comput. Geom., 33 (2005), 143–155.
T. Zhang, Sparse Recovery with Orthogonal Matching Pursuit under RIP, IEEE Trans. Inf. Theory, 57 (2011), 6215–6221.
Research was supported by NSF grant DMS-1160841.
About this article
Cite this article
Temlyakov, V.N. Dictionary descent in optimization. Anal Math 42, 69–89 (2016). https://doi.org/10.1007/s10476-016-0106-0
Key words and phrases
- Banach space
- convergence rate
Mathematics Subject Classification
- primary 41A46
- secondary 65K05, 41A65, 46B20