, Volume 62, Issue 2, pp 251–266 | Cite as

Weighted least squares fitting using ordinary least squares algorithms

  • Henk A. L. Kiers


A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.

Key words

weighted least squares alternating least squares missing data algorithms majorization matrix approximation maximum likelihood estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675.CrossRefGoogle Scholar
  2. Bijleveld, C. & de Leeuw, J. (1991). Fitting longitudinal reduced rank regression models by alternating least squares.Psychometrika, 56, 443–447.CrossRefGoogle Scholar
  3. Bollen, K. A. (1989).Structural equations with latent variables. New York: Wiley.Google Scholar
  4. Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83.PubMedGoogle Scholar
  5. Carroll, J. D., De Soete, G., & Pruzansky, S. (1989). Fitting of the latent class model via iteratively reweighted least squares candecomp with nonnegativity constraints. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 463–472). Amsterdam: Elsevier Science Publishers.Google Scholar
  6. Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.CrossRefGoogle Scholar
  7. Commandeur, J. J. F. (1991). Matching configurations. Leiden: DSWO Press.Google Scholar
  8. de Leeuw, J. & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland.Google Scholar
  9. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38.Google Scholar
  10. Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights.Technometrics, 21, 489–498.Google Scholar
  11. Gifi, A. (1990).Nonlinear multivariate analysis. Chichester: Wiley.Google Scholar
  12. Green, B. F. (1952). The orthogonal approximation of an oblique structure in factor analysis.Psychometrika, 17, 429–440.CrossRefGoogle Scholar
  13. Harman, H. H., & Jones, W. H. (1966). Factor analysis by minimizing residuals (Minres).Psychometrika, 31, 351–368.PubMedGoogle Scholar
  14. Harshman, R. A. (1978, August).Models for analysis of asymmetrical relationships among N objects or stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, Ontario.Google Scholar
  15. Harshman, R. A., Green, P. E., Wind, Y., & Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research,Marketing Science, 1, 205–242.Google Scholar
  16. Harshman, R. A., & Lundy, M. E. (1984). The PARAFAC model for three-way factor analysis and multidimensional scaling. In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.),Research methods for multimode data analysis (pp. 122–215). New York: Praeger.Google Scholar
  17. Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356.CrossRefGoogle Scholar
  18. Heiser, W. J. (1995). Convergent computation by iterative majorization: theory and applications in multidimensional data analysis. In W. J. Krzanowski (Ed.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.Google Scholar
  19. Jöreskog, K. G., & Sörbom, D. (1993).LISREL 8 User's guide. Chicago: Scientific Software International.Google Scholar
  20. Kiers, H. A. L. (1989). An alternating least squares algorithm for fitting the two- and three-way DEDICOM model and the IDIOSCAL model.Psychometrika, 54, 515–521.Google Scholar
  21. Kiers, H. A. L. (1990). Majorization as a tool for optimizing a class of matrix functions.Psychometrika, 55, 417–428.Google Scholar
  22. Kiers, H. A. L. (1993). An alternating least squares algorithm for PARAFAC2 and DEDICOM3.Computational Statistics and Data Analysis, 16, 103–118.CrossRefGoogle Scholar
  23. Kiers, H. A. L., & ten Berge, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization.Psychometrika, 57, 371–382.Google Scholar
  24. Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane's algorithm for DEDICOM.Psychometrika, 55, 151–158.Google Scholar
  25. Takane, Y. (1985). Diagonal Estimation in DEDICOM.Proceedings of the 1985 Annual Meeting of the Behaviormetric Society (pp. 100–101). Sapporo, Japan: Behaviormetric Society.Google Scholar
  26. ten Berge, J. M. F., & Kiers, H. A. L. (1989). Fitting the off-diagonal DEDICOM model in the least-squares sense by a generalization of the Harman & Jones MINRES procedure of factor analysis.Psychometrika, 54, 333–337.Google Scholar
  27. ten Berge, J. M. F., & Kiers, H. A. L. (1993). An alternating least squares method for the weighted approximation of a symmetric matrix.Psychometrika, 58, 115–118.CrossRefGoogle Scholar
  28. ten Berge, J. M. F., Kiers, H. A. L., & Commandeur, J. J. F. (1993). Orthogonal Procrustes rotation for matrices with missing values.British Journal of Mathematical and Statistical Psychology, 46, 119–134.Google Scholar
  29. Verboon, P. (1994).A robust approach to nonlinear multivariate analysis. Leiden: DSWO Press.Google Scholar
  30. Verboon, P., & Heiser, W. J. (1992). Resistant orthogonal Procrustes analysis.Journal of Classification, 9, 237–256.Google Scholar
  31. Verboon, P., & Heiser, W. J. (1994). Resistant lower rank approximation of matrices by iterative majorization.Computational Statistics and Data Analysis, 18, 457–467.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 1997

Authors and Affiliations

  • Henk A. L. Kiers
    • 1
  1. 1.University of GroningenThe Netherlands

Personalised recommendations