The Weighted Generalized Inverse in Nonlinear Programming — Active Set Selection Using a Variable-Metric Generalization of the Simplex Algorithm

Abstract

Murtagh and Saunders have developed an extension of the revised simplex method which is used in their general purpose production code, MINOS, to implement a variable-metric algorithm for linearly constrained optimization. The algorithm used in MINOS employs an approximation to the matrix ${{Z}^{T}}\tilde{G}Z$ which, if exact, would be used in computing the descent direction $-P{{\tilde{G}}^{-1}}\nabla F\left( {\tilde{x}} \right)$ where, in the idempotent projection matrix $P=Z{{\left( {{Z}^{T}}\tilde{G}Z \right)}^{-1}}{{Z}^{T}}\tilde{G}=I-{{\tilde{G}}^{-1}}{{\tilde{A}}^{T}}{{\left( \tilde{A}{{{\tilde{G}}}^{-1}}{{{\tilde{A}}}^{T}} \right)}^{-1}}\tilde{A}$, Ã is a matrix of active constraint normals; $\tilde{G}$ is the Hessian matrix (assumed positive definite) of the nonlinear objective function, F(x), evaluated at the point $\tilde{x}$; ∇F(x) is the gradient of F(x); and Z is a matrix whose linearly independent columns form a basis for the null space of Ã.

In Section 2 of this paper it is observed that P = I − Ã^Ià where Ã^I is a weighted generalized inverse of à associated with $\tilde{G}$; thus,P is an orthogonal projection matrix in the inner product space $\And \left( {\tilde{G}} \right)$,where inner product is defined as $\left( x,y \right)\tilde{G}=\left( x,\tilde{G}y \right)$.

In Section 3 inner products in $\And \left( {\tilde{G}} \right)$ are used to define generalized reduced cost coefficients which may be employed, sequentially, in a variety of ways in selecting a sequence of facets, F₁,...,F_k, of the polytope of feasible solutions,Ω , as candidates for suboptimization. The selection scheme involves a variable-metric generalization of the simplex algorithm in a manner completely analogous to that given at IX International Symposium on Mathematical Programming for the linear programming problem.

Section 4 contains remarks relating to the evolution of these developments, which are illustrated in Section 5 with a simple numerical example taken from Luenberger.

One of the objectives of this research is to develop algorithms for selecting facets of Ω (which correspond to particular sets of active constraints) so that repeated applications of suboptimization, when restricted to the facets selected, will lead to an over-all reduction in computational effort. The approach proposed in Section 3 is being implemented as a variation of the method used in MINOS (a Fortran program consisting of approximately 8000 statements) under the sponsorship of National Science Foundation Grant MCS-77-21085.

A portion of this research was sponsored by Bell Telephone Laboratories, Holmdel, N.J., and by National Science Foundation Grant DCR-74-17282.