Skip to main content
Log in

Regularization Methods for Uniformly Rank-Deficient Nonlinear Least-Squares Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In solving the nonlinear least-squares problem of minimizing ||f(x)|| 22 , difficulties arise with standard approaches, such as the Levenberg-Marquardt approach, when the Jacobian of f is rank-deficient or very ill-conditioned at the solution. To handle this difficulty, we study a special class of least-squares problems that are uniformly rank-deficient, i.e., the Jacobian of f has the same deficient rank in the neighborhood of a solution. For such problems, the solution is not locally unique. We present two solution tecniques: (i) finding a minimum-norm solution to the basic problem, (ii) using a Tikhonov regularization. Optimality conditions and algorithms are given for both of these strategies. Asymptotical convergence properties of the algorithms are derived and confirmed by numerical experiments. Extensions of the presented ideas make it possible to solve more general nonlinear least-squares problems in which the Jacobian of f at the solution is rank-deficient or ill-conditioned.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. Eriksson (1996) Optimization and Regularization of Nonlinear Least-Squares Problems, PhD Thesis Umeå University Umeå, Sweden

    Google Scholar 

  2. E.L. Allgower K. Georg (1991) Numerical Continuation Methods Springer Verlag Berlin, Germany

    Google Scholar 

  3. H. Walker (1990) Newton-Like Methods for Underdetermined Systems, Lectures in Applied Mathematics: Computational Solution of Nonlinear Systems of Equations E. Allgower K. Georg (Eds) American Mathematical Society. Providence Rhode Island 679–699

    Google Scholar 

  4. P. Lemmerling (1999) Structured Total Least Squares: Analysis, Algorithms, and Applications, PhD Thesis Katholieke Universiteit Lueven, Belgium

    Google Scholar 

  5. A. Pruessner D.P. O’Leary (2003) ArticleTitleBlind Deconvolution Using a Regularized Structured Total Least-Norm Algorithm SIAM Journal on Matrix Analysis and Applications 24 1018–1037 Occurrence Handle10.1137/S0895479801395446

    Article  Google Scholar 

  6. Eriksson, J., and Gulliksson, M., Local Results for the Gauss-Newton Method on Constrained Exactly Rank-Deficient Nonlinear Least Squares, Technical Report UMINF 97.12, Department of Computing Science, Umeå University, Umeå, Sweden, 1997.

  7. P.E. Gill W. Murray (1978) ArticleTitleAlgorithms for the Solution of the Nonlinear Least–Squares Problem SIAM Journal on Numerical Analysis 15 976–992 Occurrence Handle10.1137/0715063

    Article  Google Scholar 

  8. P. Frank R.B. Schnabel (1984) ArticleTitleTensor Methods for Nonlinear Equations SIAM Journal on Numerical Analysis 21 815–843 Occurrence Handle10.1137/0721054

    Article  Google Scholar 

  9. H. Engl M. Hanke A. Neubauer (1996) Regularization of Inverse Problems Kluwer Dordrecht, Netherlands

    Google Scholar 

  10. A.N. Tikhonov A. Goncharsky V.V. Stepanov A.G. Yagola (1995) Numerical Methods for the Solution of Ill-Posed Problems Kluwer Academic Publishers Dordrecht, Netherlands

    Google Scholar 

  11. L. Conlar (1993) Differential Manifolds: A First Course Birkhauser Advanced Texts Basel, Switzerland

    Google Scholar 

  12. Eriksson, J., Lindström, P., and Wedin, P. A., A New Regularization Method for Rank-Deficient Nonlinear Least Squares, Technical Report UMINF-95.01, Department of Computing Science, University of Umeå, Umeå, Sweden, 1995.

  13. P.E. Gill W. Murray M.H. Wright (1982) Practical Optimization Academic Press London, United Kingdom

    Google Scholar 

  14. D.P. Bertsekas (1995) Nonlinear Programming Athena Scientific Belmont, Massachusetts

    Google Scholar 

  15. Moré, J. J., The Levenberg-Marquardt Algorithm: Implementation and Theory, Proceedings of the 1977 Dundee Conference on Numerical Analysis, Edited by G. A. Watson, Springer Verlag, Berlin, Germany, pp. 105–116, 1978.

  16. Deuflhard, P., and Apostolescu, V., An Underrelaxed Gauss-Newton Method for Equality Constrained Nonlinear Least Squares, Proceedings of the IFIP Conference on Optimization Techniques, Edited by Balakrishnan and Thoma, Springer Verlag, Berlin, Germany, pp. 22–32, 1978.

  17. L.C.W. Dixon D.J. Mills (1992) ArticleTitleNeural Network and Nonlinear Optimization, Part 1: The Representation of Continuous Functions Optimization Methods and Software 1 141–151

    Google Scholar 

  18. Lindström P., Wedin P.A., Methods and Software for Nonlinear Least-Squares Problems, Technical Report UMINF–133.87, Institution of Information Processing, University of Umeå, Umeå, Sweden, 1988.

  19. Eriksson, J., and Wedin, P. A., Regularization Methods for Nonlinear Least Squares. Part 1: Exactly Rank-deficient Problems, Technical Report UMINF-96.03, Department of Computing Science, Umeå University, Umeå, Sweden, 1996.

  20. M.E. Gulliksson P.A. Wedin (2000) ArticleTitleThe Use and Properties of Tikhonov Filter Matrices SIAM Journal on Matrix Analysis and Applications 22 276–281 Occurrence Handle10.1137/S0895479899355025

    Article  Google Scholar 

  21. J.M. Ortega W.C. Rheinboldt (1970) Iterative Solution of Nonlinear Equations in Several Variables Academic Press New York, NY

    Google Scholar 

  22. H. Ramsin P.A. Wedin (1977) ArticleTitleA Comparison of Some Algorithms for the Nonlinear Least-Squares Problem, BIT 17 72–90

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

The authors would like to thank Prof. Margaret H. Wright for greatly improving the readability of the manuscript.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eriksson, J., Wedin, P.A., Gulliksson, M.E. et al. Regularization Methods for Uniformly Rank-Deficient Nonlinear Least-Squares Problems. J Optim Theory Appl 127, 1–26 (2005). https://doi.org/10.1007/s10957-005-6389-0

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-005-6389-0

Keywords

Navigation