Abstract
This paper extends prior work by the authors on solving nonlinear least squares unconstrained problems using a factorized quasi-Newton technique. With this aim we use a primal-dual interior-point algorithm for nonconvex nonlinear programming. The factorized quasi-Newton technique is now applied to the Hessian of the Lagrangian function for the transformed problem which is based on a logarithmic barrier formulation. We emphasize the importance of establishing and maintaining symmetric quasi-definiteness of the reduced KKT system. The algorithm then tries to choose a step size that reduces a merit function, and to select a penalty parameter that ensures descent directions along the iterative process. Computational results are included for a variety of least squares constrained problems and preliminary numerical testing indicates that the algorithm is robust and efficient in practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Al-Baali M. (1991). A Rule for Comparing Two Methods in Practical Optimization. Report 119. University of Damascus.
Armand P., Gilbert J.C. and Jan-Jégou S. (2000). A Feasible BFGS Interior Point Algorithm for Solving Convex Minimization Problems.SIAM Journal on Optimization 11, 199–222.
Benson H.Y., Shanno D.F. and Vanderbei R.J. (2004). Interior-Point Methods for Nonconvex Nonlinear Programming: Jamming and Numerical Testing.Mathematical Programming A 99, 35–48.
Bongartz I., Conn A. R., Gould N. and Toint Ph.L. (1995). CUTE: Constrained and Unconstrained Testing Environment.ACM Transactions on Mathematical Software 21, 123–160.
Broyden C.G., Dennis J.E. and Moré J.J. (1973). On the Local and Superlinear Convergence of Quasi-Newton Methods.Journal of Institute of Mathematics and its Applications 12, 223–245.
Byrd R.H., Hribar M.E. and Nocedal J. (1999). An Interior Point Algorithm for Large-Scale Nonlinear Programming.SIAM Journal on Optimization 9, 877–900.
Byrd R.H., Gilbert J.C. and Nocedal J. (2000). A Trust Region Method Based on Interior Point Techniques for Nonlinear Programming.Mathematical Programming 89, 149–185.
Costa M.F.P. (2002). Técnicas Quasi-Newton Factorizadas em Problemas de Mínimos Quadrados Não Lineares. Ph.D. Thesis, University of Minho.
Costa M.F.P. and Fernandes E.M.G.P. (2003). Factorized Quasi-Newton Formulae for Least Squares Problems. Technical Report, University of Minho.
Dennis J.E. and Moré J.J. (1974). A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods.Mathematics of Computation 28, 549–560.
Dennis J.E., Gay D.M. and Welsch R.E. (1981). An Adaptive Nonlinear Least-Squares Algorithm.ACM Transactions on Mathematical Software 7, 348–368.
Dennis J.E. and Schnabel R.B. (1983).Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall.
El-Bakry A.S., Tapia R. A., Tsuchiya T. and Zhang Y. (1996). On the Formulation and Theory of Newton Interior-Point Method for Nonlinear Programming.Journal of Optimization Theory and Applications 89, 507–541.
Fiacco A.V. and McCormick G.P. (1990).Nonlinear Programming. Sequential Unconstrained Minimization Techniques (second edition). SIAM.
Forsgren A.L. and Gill P.E. (1998). Primal-Dual Interior Methods for Nonconvex Nonlinear Programming.SIAM Journal on Optimization 8, 1132–1152.
Gay D.M., Overton M.L. and Wright M.H. (1997). A Primal-Dual Interior Method for Nonconvex Nonlinear Programming. Technical Report 97-4-08, Computing Sciences Research, Bell Laboratories, Murray Hill, NJ.
Gill P.E. and Murray W. (1976). Algorithms for the Solution of Nonlinear Least-Lquares Problems. Technical Report 71, Division of Numerical Analysis and Computing, National Physical Laboratory.
Hock W. and Schittkowski K. (1981).Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems 187. Springer Verlag.
Kelly C.T. (1999).Iterative Methods for Optimization. SIAM.
McKeown J.J. (1972). Experience with Peckham’s Algorithm for Minimising Sums of Squared Functions. Technical Report 29, Numerical Optimisation Centre, the Haltfield Polytechnic.
Moré J.J. (1978). The Levenberg-Marquardt Algorithm Implementation and Theory. In: Watson G.A. (ed.),Numerical Analysis Proceedings. Lectures Notes in Mathematics 630. Springer Verlag, 105–116.
Nazareth L. (1980). Some Recent Approaches to Solving Large Residual Nonlinear Least Squares Problems.SIAM Review 24, 1–11.
Nocedal J. and Wright S.J. (2000).Numerical Optimization (second edition). Springer Series in Operations Research. Springer Verlag.
Shanno D.F. and Vanderbei R.J. (2000). Interior-Point Methods for Nonconvex Nonlinear Programming: Orderings and Higher-Order Methods.Mathematical Programming B 87, 303–316.
Vanderbei R.J. (1998). LOQO: An Interior-Point Code for Quadratic Programming. Technical Report SOR 94-15 (revised). Princeton University.
Vanderbei R.J. and Shanno D.F. (1999). An Interior-Point Algorithm for Nonconvex Nonlinear Programming.Computational Optimization and Applications 13, 231–252.
Wolfe M.A. (1978).Numerical Methods for Unconstrained Optimization. Van Nostrand Reinhold Company.
Yabe H. and Takahashi T. (1991). Factorized Quasi-Newton Methods for Nonlinear Least-Squares Problems.Mathematical Programming 51, 75–100.
Yamashita H. and Yabe H. (1996). Superlinear and Quadratic Convergence of Some Primal-Dual Interior Point Methods for Constrained Optimization.Mathematical Programming 75, 377–397.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fernanda, M., Costa, P. & Fernandes, E.M.G.P. A primal-dual interior-point algorithm for nonlinear least squares constrained problems. Top 13, 145–166 (2005). https://doi.org/10.1007/BF02578992
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF02578992