On the Properties of the Method of Minimization for Convex Functions with Relaxation on the Distance to Extremum
- 7 Downloads
We present a subgradient method of minimization, similar to the method of minimal iterations for solving systems of equations, which inherits from the latter convergence properties on quadratic functions. The proposed algorithm, for a certain set of parameters, coincides with the previously known method of minimizing piecewise linear functions and is an element of the family of minimization methods with relaxation of the distance to extremum, developed by B.T. Polyak, where the step length is calculated based on the predefined minimum value of the function. We link parameters of this method to the constraint on the degree of homogeneity of the function and obtain estimates on its convergence rate on convex functions. We prove that on some classes of functions it converges at the rate of a geometric progression. We also discuss the computational capabilities of this approach for solving problems with high dimension.
Keywordssubgradient convex function linear algebra minimum of a function convergence rate
Unable to display preview. Download preview PDF.
- 1.Shor, N.Z., An Application of Gradient Descent to Solve a Network Transportation Problem, Proc. Seminar on Theory and Applied Prob. Cyber. Oper. Research, Kiev: Nauch. Sovet po Kibernetike AN USSR, 1962, no. 1, pp. 9–17.Google Scholar
- 2.Polyak, B.T., One General Approach to Solving Extreme Problems, Dokl. AN USSR, 1967, vol. 174, no. 1, pp. 33–36.Google Scholar
- 6.Nemirovskii, A.S. and Yudin, D.B., Slozhnost’ zadach i effektivnost’ metodov optimizatsii (Complexity of Problems and Efficiency of Optimization Methods), Moscow: Nauka, 1979.Google Scholar
- 9.Krutikov, V.N. and Gorskaya, T.A., A Family of Relaxation Subgradient Methods with Two-Rank Correction of the Metric Matrices, Ekonom. Mat. Metody, 2009, vol. 45, no. 4, pp. 37–80.Google Scholar
- 11.Krutikov, V.N. and Vershinin, Ya.N., A Multistep Subgradient Method for Solving Nonsmooth Minimization Problems of High Dimension, Vestn. Tomsk. Gos. Univ., Mat. Mekh., 2014, no. 3, pp. 5–19.Google Scholar
- 12.Krutikov, V.N. and Vershinin, Ya.N., A Subgradient Minimization Method with Correction of Descent Vectors Based on Pairs of Training Relations, Vestn. Kemer. Gos. Univ., 2014, no. 1–1 (57), pp. 46–54.Google Scholar
- 16.Gasnikov, A.V. and Nesterov, Yu.E., A Universal Method for Stochastic Composite Optimization Problems, e-print, 2016. https://arxiv.org/ftp/arxiv/papers/1604/1604.05275.pdfGoogle Scholar
- 17.Nesterov, Yu., Subgradient Methods for Huge-Scale Optimization Problems, Math. Program., Ser. A, 2013, vol. 146, no. 1–2, pp. 275–297.Google Scholar
- 18.Polyak, B.T., Minimzation of Non-Smooth Functionals, Zh. Vychisl. Mat. Mat. Fiz., 1969, vol. 9, no. 3, pp. 507–521.Google Scholar
- 19.Samoilenko, N.S., Krutikov, V.N., and Meshechkin, V.V., A Study of One Variation of the Subgradient Method, Vestn. Kemer. Gos. Univ., 2015, no. 5 (2), pp. 55–58.Google Scholar
- 20.Fadeev, D.K. and Fadeeva, D.K., Vychislitel’nye metody lineinoi algebry (Computational Methods of Linear Algebra), Moscow: Fizmatgiz, 1963.Google Scholar