A variable projection method for solving separable nonlinear least squares problems


Consider the separable nonlinear least squares problem of findinga εR n and α εR k which, for given data (y i ,t i ),i=1,2,...m, and functions ϕ j (α,t),j=1,2,...,n(m>n), minimize the functional

$$r(a,\alpha ) = \left\| {y - \Phi (\alpha )a} \right\|_2^2$$

where θ(α) ij j (α,t i ). Golub and Pereyra have shown that this problem can be reduced to a nonlinear least squares problem involvingα only, and a linear least squares problem involvinga only. In this paper we propose a new method for determining the optimalα which computationally has proved more efficient than the Golub-Pereyra scheme.

This is a preview of subscription content, access via your institution.


  1. 1.

    Å. Björk and G. H. Golub,Iterative Refinement of linear Least Squares Solution by Householder Transformations, BIT 7, 1967.

  2. 2.

    P. E. Gill and W. Murray,Quasi Newton Methods for Unconstrained Optimization, J. Inst. Maths. Applics. 9, 1972.

  3. 3.

    G. H. Golub,Matrix Decompositions and Statistical Calculations, Statistical Computation, Roy C. Milton and John A. Nelder, eds., Academic Press, New York, 1969.

    Google Scholar 

  4. 4.

    G. H. Golub and V. Pereyra,The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems whose Variables Separate, SIAM J. Numer. Anal. 10, April 1973.

  5. 5.

    F. T. Krogh,Efficient Implementation of a Variable Projection Algorithm for Nonlinear Least Squares Problems, Comm. ACM 17, March 1974.

  6. 6.

    D. Marquardt,An Algorithm for Least Squares Estimation of Nonlinear Parameters, J. Soc. Indust. Appl. Math. 11, June 1963.

  7. 7.

    M. R. Osborne,A Class of Nonlinear Regression Problems, Data Representation pp. 94–101, R. S. Anderssen and M. R. Osborne eds., Queensland Univ. Press, 1970.

  8. 8.

    G. Peters and J. H. Wilkinson,The least squares problem and pseudo-inverses, Comp. J. 13.3, Aug. 1970, pp. 309–316.

  9. 9.

    R. C. Rao and S. K. Mitra,Generalized Inverse of Matrices and its Applications, John Wiley, New York, 1971.

    Google Scholar 

  10. 10.

    A. Ruhe and P. Å. Wedin,Algorithms for Separable Nonlinear Least Squares Problems, Stanford Computer Science Technical Report 434, July 1974.

Download references

Author information



Rights and permissions

Reprints and Permissions

About this article

Cite this article

Kaufman, L. A variable projection method for solving separable nonlinear least squares problems. BIT 15, 49–57 (1975). https://doi.org/10.1007/BF01932995

Download citation


  • Computational Mathematic
  • Projection Method
  • Variable Projection
  • Variable Projection Method