Skip to main content
Log in

Weighted least squares fitting using ordinary least squares algorithms

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675.

    Article  Google Scholar 

  • Bijleveld, C. & de Leeuw, J. (1991). Fitting longitudinal reduced rank regression models by alternating least squares.Psychometrika, 56, 443–447.

    Article  Google Scholar 

  • Bollen, K. A. (1989).Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  • Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83.

    PubMed  Google Scholar 

  • Carroll, J. D., De Soete, G., & Pruzansky, S. (1989). Fitting of the latent class model via iteratively reweighted least squares candecomp with nonnegativity constraints. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 463–472). Amsterdam: Elsevier Science Publishers.

    Google Scholar 

  • Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.

    Article  Google Scholar 

  • Commandeur, J. J. F. (1991). Matching configurations. Leiden: DSWO Press.

    Google Scholar 

  • de Leeuw, J. & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland.

    Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38.

    Google Scholar 

  • Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights.Technometrics, 21, 489–498.

    Google Scholar 

  • Gifi, A. (1990).Nonlinear multivariate analysis. Chichester: Wiley.

    Google Scholar 

  • Green, B. F. (1952). The orthogonal approximation of an oblique structure in factor analysis.Psychometrika, 17, 429–440.

    Article  Google Scholar 

  • Harman, H. H., & Jones, W. H. (1966). Factor analysis by minimizing residuals (Minres).Psychometrika, 31, 351–368.

    PubMed  Google Scholar 

  • Harshman, R. A. (1978, August).Models for analysis of asymmetrical relationships among N objects or stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, Ontario.

  • Harshman, R. A., Green, P. E., Wind, Y., & Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research,Marketing Science, 1, 205–242.

    Google Scholar 

  • Harshman, R. A., & Lundy, M. E. (1984). The PARAFAC model for three-way factor analysis and multidimensional scaling. In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.),Research methods for multimode data analysis (pp. 122–215). New York: Praeger.

    Google Scholar 

  • Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356.

    Article  Google Scholar 

  • Heiser, W. J. (1995). Convergent computation by iterative majorization: theory and applications in multidimensional data analysis. In W. J. Krzanowski (Ed.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.

    Google Scholar 

  • Jöreskog, K. G., & Sörbom, D. (1993).LISREL 8 User's guide. Chicago: Scientific Software International.

    Google Scholar 

  • Kiers, H. A. L. (1989). An alternating least squares algorithm for fitting the two- and three-way DEDICOM model and the IDIOSCAL model.Psychometrika, 54, 515–521.

    Google Scholar 

  • Kiers, H. A. L. (1990). Majorization as a tool for optimizing a class of matrix functions.Psychometrika, 55, 417–428.

    Google Scholar 

  • Kiers, H. A. L. (1993). An alternating least squares algorithm for PARAFAC2 and DEDICOM3.Computational Statistics and Data Analysis, 16, 103–118.

    Article  Google Scholar 

  • Kiers, H. A. L., & ten Berge, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization.Psychometrika, 57, 371–382.

    Google Scholar 

  • Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane's algorithm for DEDICOM.Psychometrika, 55, 151–158.

    Google Scholar 

  • Takane, Y. (1985). Diagonal Estimation in DEDICOM.Proceedings of the 1985 Annual Meeting of the Behaviormetric Society (pp. 100–101). Sapporo, Japan: Behaviormetric Society.

    Google Scholar 

  • ten Berge, J. M. F., & Kiers, H. A. L. (1989). Fitting the off-diagonal DEDICOM model in the least-squares sense by a generalization of the Harman & Jones MINRES procedure of factor analysis.Psychometrika, 54, 333–337.

    Google Scholar 

  • ten Berge, J. M. F., & Kiers, H. A. L. (1993). An alternating least squares method for the weighted approximation of a symmetric matrix.Psychometrika, 58, 115–118.

    Article  Google Scholar 

  • ten Berge, J. M. F., Kiers, H. A. L., & Commandeur, J. J. F. (1993). Orthogonal Procrustes rotation for matrices with missing values.British Journal of Mathematical and Statistical Psychology, 46, 119–134.

    Google Scholar 

  • Verboon, P. (1994).A robust approach to nonlinear multivariate analysis. Leiden: DSWO Press.

    Google Scholar 

  • Verboon, P., & Heiser, W. J. (1992). Resistant orthogonal Procrustes analysis.Journal of Classification, 9, 237–256.

    Google Scholar 

  • Verboon, P., & Heiser, W. J. (1994). Resistant lower rank approximation of matrices by iterative majorization.Computational Statistics and Data Analysis, 18, 457–467.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Additional information

This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kiers, H.A.L. Weighted least squares fitting using ordinary least squares algorithms. Psychometrika 62, 251–266 (1997). https://doi.org/10.1007/BF02295279

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02295279

Key words

Navigation