Skip to main content
SpringerLink
Log in

Adaptive Algorithm for Constrained Least-Squares Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

This paper is concerned with the implementation and testing of an algorithm for solving constrained least-squares problems. The algorithm is an adaptation to the least-squares case of sequential quadratic programming (SQP) trust-region methods for solving general constrained optimization problems. At each iteration, our local quadratic subproblem includes the use of the Gauss–Newton approximation but also encompasses a structured secant approximation along with tests of when to use this approximation. This method has been tested on a selection of standard problems. The results indicate that, for least-squares problems, the approach taken here is a viable alternative to standard general optimization methods such as the Byrd–Omojokun trust-region method and the Powell damped BFGS line search method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. FLETCHER, R., Practical Methods of Optimization, 2nd Edition, John Wiley and Sons, Chichester, England, 1987.

    Google Scholar 

  2. BYRD, R. H., Robust Trust-Region Methods for Constrained Optimization, 3rd SIAM Conference on Optimization, Houston, Texas, 1987.

  3. OMOJOKUN, E. O., Trust-Region Algorithms for Optimization with Nonlinear Equality and Inequality Constraints, PhD Thesis, University of Colorado, Boulder, Colorado, 1989.

    Google Scholar 

  4. LALEE, M., NOCEDAL, J., and PLANTENGA, T., On the Implementation of an Algorithm for Large-Scale Equality Constrained Optimization, SIAM Journal of Optimization, Vol. 8, pp. 682-706, 1998.

    Google Scholar 

  5. DENNIS, J. E., JR., GAY, D.M., and WELSCH, R. E., An Adaptiûe Nonlinear Least-Squares Algorithm, Transactions on Mathematical Software, Vol. 7, pp. 348-368, 1981.

    Article  Google Scholar 

  6. POWELL, M. J. D., A Fast Algorithm for Nonlinearly Constrained Optimization Calculation, Numerical Analysis, Edited by G. A. Watson, Lecture Notes in Mathematics, Springer Verlag, Berlin, Germany, Vol. 630, pp. 144-157, 1978.

    Google Scholar 

  7. MAHDAVI-AMIRI, N., and BARTELS, R. H., Constrained Nonlinear Least Squares: An Exact Penalty Approach with Projected Structured Quasi-Newton Updates, Transactions on Mathematical Software, Vol. 15, pp. 220-242, 1989.

    Google Scholar 

  8. FLETCHER, R., and XU, C., Hybrid Methods for Nonlinear Least Squares, IMA Journal of Numerical Analysis, Vol. 7, pp. 371-389, 1987.

    Google Scholar 

  9. TJOA, I. B., and BIEGLER, L. T., Simultaneous Solution and Optimization Strategies for Parameter Estimation of Differential-Algebraic Equation Systems, Industrial Engineering Chemical Research, Vol. 30, pp. 376-385, 1991.

    Google Scholar 

  10. TAPIA, R. A., On Secant Updates for Use in General Constrained Optimization, Mathematics of Computation, Vol. 51, pp. 181-203, 1988.

    Google Scholar 

  11. HUSCHENS, J., Exploiting Additional Structure in Equality Constrained Optimization by Structured SQP Secant Algorithms, Journal of Optimization Theory and Applications, Vol. 77, pp. 382-359, 1993.

    Google Scholar 

  12. OSBORNE, M. R., Scoring with Constraints, ANZIAM Journal, Vol. 42, pp. 9-25, 2000.

    Google Scholar 

  13. ENGELS, J. R., and MARTINEZ, H. J., Local and Superlinear Convergence for Partially Known Quasi-Newton Methods, SIAM Journal on Optimization, Vol. 1, pp. 42-56, 1991.

    Google Scholar 

  14. DENNIS, J. E., JR., and SCHNABEL, R. B., Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey, 1983.

    Google Scholar 

  15. HOCK, W., and SCHITTKOWSKI, K., Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems, Springer Verlag, Berlin, Germany, Vol. 187, 1981.

    Google Scholar 

  16. SCHITTKOWSKI, K., More Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems, Springer Verlag, Berlin, Germany, Vol. 282, 1987.

    Google Scholar 

  17. BYRD, R. H., TAPIA, R. A., and ZHANG, Y., An SQP Augmented Lagrangian BFGS Algorithm for Constrained Optimization, SIAM Journal of Optimization, Vol. 2, pp. 21-241, 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Centre for Epidemiology and Population Health, Australian National University, Canberra, Australia

    Z.F. Li (Postdoctoral Fellow)

  2. School of Mathematical Sciences, Australian National University, Canberra, Australia

    M.R. Osborne (Professor)

  3. School of Mathematics and Statistics, University of Canberra, Canberra, Australia

    T. Prvan (Senior Lecturer)

Authors
  1. Z.F. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M.R. Osborne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. T. Prvan
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Li, Z., Osborne, M. & Prvan, T. Adaptive Algorithm for Constrained Least-Squares Problems. Journal of Optimization Theory and Applications 114, 423–441 (2002). https://doi.org/10.1023/A:1016043919978

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1016043919978

Search

Navigation