Abstract
In this paper, we investigate an inexact quasisubgradient method with extrapolation for solving a quasiconvex optimization problem with a closed, convex and bounded constraint set. We establish the convergence in objective values, iteration complexity and rate of convergence for our proposed method under Hölder condition and weak sharp minima condition. When both diminishing stepsize and extrapolation stepsize are decaying as a power function, we obtain explicit iteration complexities. When diminishing stepsize is decaying as a power function and the extrapolation stepsize is decreasing not less than a power function, the diminishing stepsize provides a rate of convergence \({\mathcal {O}}\left( \tau ^{k^{s}}\right) (s \in (0,1))\) to an optimal solution or to a ball of the optimal solution set, which is faster than \({\mathcal {O}}\left( {1}/{k^\beta }\right) \) (for each \(\beta >0\)). With geometrically decreasing extrapolation stepsize, we obtain a linear rate of convergence to a ball of the optimal solution set for the constant stepsize and dynamic stepsize. Our numerical testing shows that the performance with extrapolation is much more efficient than that without extrapolation in terms of the number of iterations needed for reaching an approximate optimal solution.
Similar content being viewed by others
References
Alves, M.M., Eckstein, J., Geremia, M., Melo, J.G.: Relative-error inertial-relaxed inexact versions of douglas-rachford and ADMM splitting algorithms. Comput. Optim. Appl. 75(2), 389–422 (2020)
Auslender, A., Teboulle, M.: Interior gradient and epsilon-subgradient descent methods for constrained convex minimization. Math. Oper. Res. 29(1), 1–26 (2004)
Avriel, M., Diewert, W.E., Schaible, S., Zang, I.: Generalized Concavity. Plenum Press, New York (1988)
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183–202 (2009)
Bertsekas, D.P., Nedic̀, A., Ozdaglar, A.: Convex Analysis and Optimization. Athena Scientific, Belmont (2003)
Boţ, R.I., Csetnek, E.R., Hendrich, C.: Inertial Douglas-Rachford splitting for monotone inclusion problems. Appl. Math. Comput. 256, 472–487 (2015)
Bradley, S.P., Frey, S.C., Jr.: Fractional programming with homogeneous functions. Oper. Res. 22(2), 350–357 (1974)
Burke, J.V., Ferris, M.C.: Weak sharp minima in mathematical programming. SIAM J. Control. Optim. 31(5), 1340–1359 (1993)
Cai, X., Teo, K.-L., Yang, X., Zhou, X.: Portfolio optimization under a minimax rule. Manage. Sci. 46(7), 957–972 (2000)
Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imag. Vis. 40(1), 120–145 (2011)
Chen, C., Chan, R.H., Ma, S., Yang, J.: Inertial proximal ADMM for linearly constrained separable convex optimization. SIAM J. Imag. Sci. 8(4), 2239–2267 (2015)
Crouzeix, J.P., Martínez-Legaz, J.E., Volle, M.: Generalized Convexity, Generalized Monotonicity: Recent Results. Kluwer Academic Publishers, Dordrecht (1998)
Cruz, J.B., Pérez, L.L., Melo, J.: Convergence of the projected gradient method for quasiconvex multiobjective optimization. Nonlinear Anal. Theory Methods Appl. 74(16), 5268–5273 (2011)
Daniilidis, A., Hadjisavvas, N., Martínez-Legaz, J.E.: An appropriate subdifferential for quasiconvex functions. SIAM J. Optim. 12(2), 407–420 (2001)
dos Santos Gromicho, J.A.: Quasiconvex Optimization and Location Theory. Kluwer Academic Publishers, Dordrecht (1998)
Ermol’ev, Y.M.: Methods of solution of nonlinear extremal problems. Cybernetics 2(4), 1–14 (1966)
Giannessi, F.: Constrained Optimization and Image Space Analysis: Volume 1: Separation of Sets and Optimality Conditions. Springer Science & Business Media, New York (2005)
Greenberg, H.J., Pierskalla, W.P.: Quasi-conjugate functions and surrogate duality. Cahiers du Centre d,Etudes de Recherche Operationelle 15, 437–448 (1973)
Hadjisavvas, N., Komlósi, S., Schaible, S.S.: Handbook of Generalized Convexity and Generalized Monotonicity. Springer-Verlag, New York (2005)
Hu, Y., Yang, X., Sim, C.K.: Inexact subgradient methods for quasi-convexoptimization problems. Eur. J. Oper. Res. 240(2), 315–327 (2015)
Hu, Y., Li, J., Yu, C.K.W.: Convergence rates of subgradient methods for quasi-convex optimization problems. Comput. Optim. Appl. 77, 183–212 (2020)
Huang, X., Yang, X.: A unified augmented Lagrangian approach to duality and exact penalization. Math. Oper. Res. 28(3), 533–552 (2003)
Jia, Z., Wu, Z., Dong, X.: An inexact proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth optimization problems. J. Inequ. Appl. 125, 1–16 (2019)
Johnstone, P.R., Moulin, P.: Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions. Comput. Optim. Appl. 67(2), 259–292 (2017)
Kiwiel, K.C.: Convergence and efficiency of subgradient methods for quasiconvex minimization. Math. Program. 90(1), 1–25 (2001)
Konnov, I.V.: On properties of supporting and quasi-supporting vectors. J. Math. Sci. 71(6), 2760–2763 (1994)
Konnov, I.V.: Estimates of the labor cost of combined relaxation methods. J. Math. Sci. 74(5), 1225–1235 (1995)
Konnov, I.V.: On convergence properties of a subgradient method. Optim. Methods Softw. 18(1), 53–62 (2003)
Langenberg, N., Tichatschke, R.: Interior proximal methods for quasiconvex optimization. J. Global Optim. 52(3), 641–661 (2012)
Maingé, P.E.: Asymptotic convergence of an inertial proximal method for unconstrained quasiconvex minimization. J. Global Optim. 45(4), 631–644 (2009)
Nedic̀, A., Bertsekas, D.P.: The effect of deterministic noise in subgradient methods. Math. Program. 125(1), 75–99 (2010)
Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, Boston (2004)
Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103(1), 127–152 (2005)
Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J. Imag. Sci. 7(2), 1388–1419 (2014)
Plastria, F.: Lower subdifferentiable functions and their minimization by cutting planes. J. Optim. Theory Appl. 46(1), 37–53 (1985)
Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (IPALM) for nonconvex and nonsmooth problems. SIAM J. Imag. Sci. 9(4), 1756–1787 (2016)
Polyak, B.T.: A general method for solving extremal problems. (Russian) Doklady Akademii Nauk, 174(1), 33–36 (1967)
Polyak, B.T.: Introduction to Optimization. Optimization Software, New York (1987)
Ramík, J., Vlach, M.: Generalized Concavity in Fuzzy Optimization and Decision Analysis. Kluwer Academic Publishers, Boston (2012)
Sharpe, W.F.: Mutual fund performance. J. Bus. 39(1), 119–138 (1966)
Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer-Verlag, New York (1985)
Wen, B., Chen, X., Pong, T.K.: Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems. SIAM J. Optim. 27(1), 124–145 (2017)
Wu, Z., Li, M.: General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems. Comput. Optim. Appl. 73(1), 129–158 (2019)
Yu, C.K.W., Hu, Y., Yang, X., Choy, S.K.: Abstract convergence theorem for quasi-convex optimization problems with applications. Optimization 68(7), 1289–1304 (2019)
Zhang, X., Barrio, R., Martínez, M.A., Jiang, H., Cheng, L.: Bregman proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems. IEEE Access 7, 126515–126529 (2019)
Acknowledgements
We would like to thank the reviewer for providing many constructive comments and suggestions and thank Professor I.V. Konnov for his suggestions on an early version of the paper, in particular to construct a nonsmooth quasiconvex optimization problem (see Sect. 6.2). These comments and suggestions have improved the presentation of the paper. The first author was supported in part by grants from the Research Grants Council of the Hong Kong Special Administrative Region, China (RGC Ref No. 15234216).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This paper is dedicated to the 85th birthday of Professor Franco Giannessi.
Communicated by: Liqun Qi.
Rights and permissions
About this article
Cite this article
Yang, X., Zu, C. Convergence of Inexact Quasisubgradient Methods with Extrapolation. J Optim Theory Appl 193, 676–703 (2022). https://doi.org/10.1007/s10957-022-02014-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-022-02014-1