Convergence rate of the gradient descent method with dilatation of the space

Conclusion

As we see from Theorem 2, if the GDDS process has the given properties, it ensures convergence to the minimum of the functional like that of a geometrical progression. In particular, the necessary properties of the GDDS process are ensured by the conditions of Theorem 1 if m=m^* (see Theorems 2, 3 of [1]). When we reduce a problem of solving a system of equations or inequalities to a minimization problem, m^* is unknown. Thus, the GDDS method must be effective in solving systems of nonlinear equations and inequalities. The essential feature of this method is that the guaranteed denominator of the geometrical progression$q = {1 \mathord{\left/ {\vphantom {1 {\sqrt[n]{\alpha }}}} \right. \kern-\nulldelimiterspace} {\sqrt[n]{\alpha }}}$ is independent of the degree of pittedness off(x), which makes it possible to successfully use it to solve systems of equations which are nearly degenerate.

As a increases the effectiveness of the method in general decreases, both in connection with the fact that 1 becomes close to unity and because a large memory is required to store the matrix B_k. The modification proposed when m^* is unknown can be used to solve minimax problems in the general problem of convex programming by reducing it to the solution of a sequence of convex inequalities or by using the method of penalty functions with expansion schemes.