Skip to main content
SpringerLink
Log in

Duality Gap Estimates for a Class of Greedy Optimization Algorithms in Banach Spaces

  • Conference paper
  • First Online:
Mathematical Optimization Theory and Operations Research (MOTOR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12755))

  • 650 Accesses

Abstract

The paper examines a class of algorithms called Weak Biorthogonal Greedy Algorithms (WBGA) designed for the task of finding the approximate solution to a convex cardinality-constrained optimization problem in a Banach space using linear combinations of some set of “simple” elements of this space (a dictionary), i.e. the problem of finding the infimum of a given convex function over all linear combinations of the dictionary elements with the given cardinality. An important issue when one computationally solves optimization problems is to obtain an estimate of the proximity to an optimal solution, that can be used to effectively check that the approximate solution is with a given accuracy. A similar idea that has already been applied to solving some optimization problems, in which such an estimate of the proximity (certificate) to the optimal solution is called the “duality gap”. We introduce the notion of the duality gap for greedy optimization in Banach Spaces and obtain dual convergence estimates for sparse-constrained optimization by means of algorithms from the WBGA class.

This work was supported by the Ministry of Science and Higher Education of the Russian Federation in the framework of the basic part of the scientific research state task, project FSRR-2020-0006.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barron, A.R., Cohen, A., Dahmen, W., DeVore, R.A.: Approximation and learning by greedy algorithms. Ann. Stat. 36(1), 64–94 (2008). https://doi.org/10.1214/009053607000000631

    Article  MathSciNet  MATH  Google Scholar 

  2. Chena, Z., Huang, C., Lin, S.: A new sparse representation framework for compressed sensing mri. Knowl. Based Syst. 188, 104969 (2020). https://doi.org/10.1016/j.knosys.2019.104969

  3. Clarkson, K.L.: Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Trans. Algorithms 6(4), 1–30 (2010). https://doi.org/10.1145/1824777.1824783

    Article  MathSciNet  MATH  Google Scholar 

  4. Davis, G., Mallat, S., Avellaneda, M.: Adaptive greedy approximation. Constr. Approx. 13(1), 57–98 (1997). https://doi.org/10.1007/bf02678430

    Article  MathSciNet  MATH  Google Scholar 

  5. Dereventsov, A., Temlyakov, V.: A unified way of analyzing some greedy algorithms. J. Funct. Anal. 277(12), 108286 (2019). https://doi.org/10.1016/j.jfa.2019.108286, http://www.sciencedirect.com/science/article/pii/S0022123619302496

  6. DeVore, R.A., Temlyakov, V.N.: Some remarks on greedy algorithms. Adv. Comput. Math. 5, 173–187 (1996). https://doi.org/10.1007/bf02124742

    Article  MathSciNet  MATH  Google Scholar 

  7. DeVore, R.A., Temlyakov, V.N.: Convex optimization on banach spaces. Found. Comput. Math. 16(2), 369–394 (2016)

    Article  MathSciNet  Google Scholar 

  8. Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006). https://doi.org/10.1109/TIT.2006.871582

    Article  MathSciNet  MATH  Google Scholar 

  9. Fan, J., Lv, J., Qi, L.: Sparse high-dimensional models in economics. Ann. Rev. Econ. 3(1), 291–317 (2011). https://doi.org/10.1146/annurev-economics-061109-080451

  10. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Naval Res. Logis. Q. 3(1–2), 95–110 (1956). https://doi.org/10.1002/nav.3800030109

  11. Freund, R.M., Grigas, P.: New analysis and results for the frank-wolfe method. Math. Program. 155(1), 199–230 (2016)

    Article  MathSciNet  Google Scholar 

  12. Friedman, J.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189–1232 (2001)

    Article  MathSciNet  Google Scholar 

  13. Huber, P.J.: Projection pursuit. Ann. Stat. 13, 435–525 (1985)

    MathSciNet  MATH  Google Scholar 

  14. Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 427–435 (2013)

    Google Scholar 

  15. Jones, L.: On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Stat. 15(2), 880–882 (1987). https://doi.org/10.1214/aos/1176350382

    Article  MathSciNet  MATH  Google Scholar 

  16. Konyagin, S.V., Temlyakov, V.N.: A remark on greedy approximation in banach spaces. East J. Approx. 5(3), 365–379 (1999)

    MathSciNet  MATH  Google Scholar 

  17. Levitin, E.S., Polyak, B.T.: Constrained minimization methods. USSR Comput. Math. Math. Phys. 6(5), 1–50 (1966)

    Article  Google Scholar 

  18. Mironov, S.V., Sidorov, S.P.: Duality gap estimates for weak Chebyshev greedy algorithms in Banach spaces. Comput. Math. Math. Phys. 59(6), 904–914 (2019). https://doi.org/10.1134/S0965542519060113

    Article  MathSciNet  MATH  Google Scholar 

  19. Nemirovski, A., Onn, S., Rothblum, U.G.: Accuracy certificates for computational problems with convex structure. Math. Oper. Res. 35(1), 52–78 (2010). https://doi.org/10.1287/moor.1090.0427

    Article  MathSciNet  MATH  Google Scholar 

  20. Nesterov, Y.: Complexity bounds for primal-dual methods minimizing the model of objective function. Math. Program. 171(1-2), 311–330 (2017). https://doi.org/10.1007/s10107-017-1188-6

  21. Nguyen, H., Petrova, G.: Greedy strategies for convex optimization. Calcolo 54(1), 207–224 (2016). https://doi.org/10.1007/s10092-016-0183-2

    Article  MathSciNet  MATH  Google Scholar 

  22. Sidorov, S.P., Mironov, S.V., Pleshakov, M.G.: Dual convergence estimates for a family of greedy algorithms in Banach spaces. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R. (eds.) MOD 2017. LNCS, vol. 10710, pp. 109–120. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-72926-8_10

    Chapter  Google Scholar 

  23. Sidorov, S.P., Faizliev, A.R., Khomchenko, A.A.: Algorithms for \(l_1\)-norm minimisation of index tracking error and their performance. Int. J. Math. Oper. Res. 11(4), 497–519 (2017). https://ideas.repec.org/a/ids/ijmore/v11y2017i4p497-519.html

  24. Sidorov, S.P., Mironov, S.V.: Duality gap analysis of weak relaxed greedy algorithms. In: Battiti, R., Kvasov, D.E., Sergeyev, Y.D. (eds.) LION 2017. LNCS, vol. 10556, pp. 251–262. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69404-7_18

    Chapter  Google Scholar 

  25. Temlyakov, V.N.: Greedy approximation in convex optimization. Constr. Approx. 41(2), 269–296 (2015). https://doi.org/10.1007/s00365-014-9272-0

    Article  MathSciNet  MATH  Google Scholar 

  26. Temlyakov, V.N.: Dictionary descent in optimization. Anal. Mathematica 42(1), 69–89 (2016). https://doi.org/10.1007/s10476-016-0106-0

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhang, Z., Shwartz, S., Wagner, L., Miller, W.: A greedy algorithm for aligning DNA sequences. J. Comput. Biol. 7(1–2), 203–214 (2000). https://doi.org/10.1089/10665270050081478

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saratov State University, Saratov, 410012, Russian Federation

    Sergei Sidorov & Kirill Spiridinov

Authors
  1. Sergei Sidorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kirill Spiridinov
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  2. Krasovsky Institute of Mathematics and Mechanics, Ekaterinburg, Russia

    Michael Khachay

  3. Matrosov Institute for System Dynamics and Control Theory, Irkutsk, Russia

    Alexander Kazakov

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sidorov, S., Spiridinov, K. (2021). Duality Gap Estimates for a Class of Greedy Optimization Algorithms in Banach Spaces. In: Pardalos, P., Khachay, M., Kazakov, A. (eds) Mathematical Optimization Theory and Operations Research. MOTOR 2021. Lecture Notes in Computer Science(), vol 12755. Springer, Cham. https://doi.org/10.1007/978-3-030-77876-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-77876-7_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-77875-0

  • Online ISBN: 978-3-030-77876-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation