Journal of Optimization Theory and Applications

, Volume 100, Issue 1, pp 145–160

KKT Conditions for Rank-Deficient Nonlinear Least-Square Problems with Rank-Deficient Nonlinear Constraints

  • M. Gulliksson


In nonlinear least-square problems with nonlinear constraints, the function \(\left. {(1/2)} \right\|\left. {f_2 (x)} \right\|_2^2\), where f2 is a nonlinear vector function, is to be minimized subject to the nonlinear constraints f1(x)=0. This problem is ill-posed if the first-order KKT conditions do not define a locally unique solution. We show that the problem is ill-posed if either the Jacobian of f1 or the Jacobian of J is rank-deficient (i.e., not of full rank) in a neighborhood of a solution satisfying the first-order KKT conditions. Either of these ill-posed cases makes it impossible to use a standard Gauss–Newton method. Therefore, we formulate a constrained least-norm problem that can be used when either of these ill-posed cases occur. By using the constant-rank theorem, we derive the necessary and sufficient conditions for a local minimum of this minimum-norm problem. The results given here are crucial for deriving methods solving the rank-deficient problem.

Nonlinear least squares optimization regularization KKT conditions rank-deficient nonlinear constraints rank-deficient nonlinear least-square problems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Walker, H. F., Newton-Like Methods for Underdetermined Systems, Lectures in Applied Mathematics, Vol. 26, pp. 679–699, 1990.Google Scholar
  2. 2.
    Bates, D., and Watts, D., Nonlinear Regression Analysis and Its Applications, John Wiley, New York, New York, 1988.Google Scholar
  3. 3.
    Van Huffel, S., and Vandewalle, J., The Total Least-Square Problem: Computational Aspects and Analysis, SIAM, Philadelphia, Pennsylvania, 1991.Google Scholar
  4. 4.
    Eriksson, J., Gulliksson, M., LindstrÖm, P., and Wedin, P. Å., Regularization Tools for Training Feed-Forward Neural Networks, Part 2: Large-Scale Problems, Technical Report UMINF 96.06, Department of Computing Science, Umeå University, Umeå, Sweden, 1996.Google Scholar
  5. 5.
    Hansen, P. C., Rank-Deficient and Discrete Ill-Posed Problems, Technical Report, Department of Mathematical Modelling, Section for Numerical Analysis, Technical University of Denmark, Lyngby, Denmark, 1997.Google Scholar
  6. 6.
    Eriksson, J., and Gulliksson, M., Local Results for the Gauss-Newton Method on Constrained Exactly Rank-Deficient Nonlinear Least Squares, Technical Report UMINF 97.12, Department of Computing Science, Umeå University, Umeå, Sweden, 1997.Google Scholar
  7. 7.
    Eriksson, J., Optimization and Regularization of Nonlinear Least-Square Problems, Technical Report UMINF 96.09 (PhD Thesis), Department of Computing Science, Umeå University, Umeå, Sweden, 1996.Google Scholar
  8. 8.
    Hanson, R. J., and Lawson, C. L., Solving Least-Square Problems, Prentice Hall, Englewood Cliffs, New Jersey, 1974.Google Scholar
  9. 9.
    Conlar, L., Differential Manifolds: A First Course, Birkhäuser Advanced Texts, Boston, Massachusetts, 1993.Google Scholar

Copyright information

© Plenum Publishing Corporation 1999

Authors and Affiliations

  • M. Gulliksson
    • 1
  1. 1.Department of Computing ScienceUmeå UniversityUmeåSweden

Personalised recommendations