Regularization Methods for Uniformly Rank-Deficient Nonlinear Least-Squares Problems Article DOI:
Cite this article as: Eriksson, J., Wedin, P.A., Gulliksson, M.E. et al. J Optim Theory Appl (2005) 127: 1. doi:10.1007/s10957-005-6389-0 Abstract
In solving the nonlinear least-squares problem of minimizing ||
f(x)|| , difficulties arise with standard approaches, such as the Levenberg-Marquardt approach, when the Jacobian of 2 2 f is rank-deficient or very ill-conditioned at the solution. To handle this difficulty, we study a special class of least-squares problems that are uniformly rank-deficient, i.e., the Jacobian of f has the same deficient rank in the neighborhood of a solution. For such problems, the solution is not locally unique. We present two solution tecniques: (i) finding a minimum-norm solution to the basic problem, (ii) using a Tikhonov regularization. Optimality conditions and algorithms are given for both of these strategies. Asymptotical convergence properties of the algorithms are derived and confirmed by numerical experiments. Extensions of the presented ideas make it possible to solve more general nonlinear least-squares problems in which the Jacobian of f at the solution is rank-deficient or ill-conditioned. Keywords Nonlinear least squares Gauss-Newton Method rank-deficient matrices minimum norm problems truncation problems stabilization methods ill-posed problems Tikhonov regularization
The authors would like to thank Prof. Margaret H. Wright for greatly improving the readability of the manuscript.
Eriksson, J. 1996Optimization and Regularization of Nonlinear Least-Squares Problems, PhD Thesis Umeå University Umeå, Sweden Google Scholar
Allgower, E.L., Georg, K. 1991Numerical Continuation Methods Springer Verlag Berlin, Germany Google Scholar
Walker, H. 1990 Newton-Like Methods for Underdetermined Systems, Lectures in Applied Mathematics: Computational Solution of Nonlinear Systems of Equations Allgower, E. Georg, K. eds. American Mathematical Society. Providence Rhode Island 679 699 Google Scholar
Lemmerling, P. 1999Structured Total Least Squares: Analysis, Algorithms, and Applications, PhD Thesis Katholieke Universiteit Lueven, Belgium Google Scholar
Pruessner, A., O’Leary, D.P. 2003 Blind Deconvolution Using a Regularized Structured Total Least-Norm Algorithm SIAM Journal on Matrix Analysis and Applications 24 1018 1037 CrossRef Google Scholar
Eriksson, J., and Gulliksson, M., Local Results for the Gauss-Newton Method on Constrained Exactly Rank-Deficient Nonlinear Least Squares, Technical Report UMINF 97.12, Department of Computing Science, Umeå University, Umeå, Sweden, 1997.
Gill, P.E., Murray, W. 1978 Algorithms for the Solution of the Nonlinear Least–Squares Problem SIAM Journal on Numerical Analysis 15 976 992 CrossRef Google Scholar
Frank, P., Schnabel, R.B. 1984 Tensor Methods for Nonlinear Equations SIAM Journal on Numerical Analysis 21 815 843 CrossRef Google Scholar
Engl, H., Hanke, M., Neubauer, A. 1996Regularization of Inverse Problems Kluwer Dordrecht, Netherlands Google Scholar
Tikhonov, A.N., Goncharsky, A., Stepanov, V.V., Yagola, A.G. 1995Numerical Methods for the Solution of Ill-Posed Problems Kluwer Academic Publishers Dordrecht, Netherlands Google Scholar
Conlar, L. 1993Differential Manifolds: A First Course Birkhauser Advanced Texts Basel, Switzerland Google Scholar
Eriksson, J., Lindström, P., and Wedin, P. A., A New Regularization Method for Rank-Deficient Nonlinear Least Squares, Technical Report UMINF-95.01, Department of Computing Science, University of Umeå, Umeå, Sweden, 1995.
Gill, P.E., Murray, W., Wright, M.H. 1982Practical Optimization Academic Press London, United Kingdom Google Scholar
Bertsekas, D.P. 1995Nonlinear Programming Athena Scientific Belmont, Massachusetts Google Scholar
Moré, J. J., The Levenberg-Marquardt Algorithm: Implementation and Theory, Proceedings of the 1977 Dundee Conference on Numerical Analysis, Edited by G. A. Watson, Springer Verlag, Berlin, Germany, pp. 105–116, 1978.
Deuflhard, P., and Apostolescu, V., An Underrelaxed Gauss-Newton Method for Equality Constrained Nonlinear Least Squares, Proceedings of the IFIP Conference on Optimization Techniques, Edited by Balakrishnan and Thoma, Springer Verlag, Berlin, Germany, pp. 22–32, 1978.
Dixon, L.C.W., Mills, D.J. 1992 Neural Network and Nonlinear Optimization, Part 1: The Representation of Continuous Functions Optimization Methods and Software 1 141 151 Google Scholar
Lindström P., Wedin P.A., Methods and Software for Nonlinear Least-Squares Problems, Technical Report UMINF–133.87, Institution of Information Processing, University of Umeå, Umeå, Sweden, 1988.
Eriksson, J., and Wedin, P. A., Regularization Methods for Nonlinear Least Squares. Part 1: Exactly Rank-deficient Problems, Technical Report UMINF-96.03, Department of Computing Science, Umeå University, Umeå, Sweden, 1996.
Gulliksson, M.E., Wedin, P.A. 2000 The Use and Properties of Tikhonov Filter Matrices SIAM Journal on Matrix Analysis and Applications 22 276 281 CrossRef Google Scholar
Ortega, J.M., Rheinboldt, W.C. 1970Iterative Solution of Nonlinear Equations in Several Variables Academic Press New York, NY Google Scholar
Ramsin, H., Wedin, P.A. 1977 A Comparison of Some Algorithms for the Nonlinear Least-Squares Problem, BIT 17 72 90 Google Scholar Copyright information
© Springer Science+Business Media, Inc. 2005