, Volume 55, Issue 3, pp 417–428 | Cite as

Majorization as a tool for optimizing a class of matrix functions

  • Henk A. L. Kiers


The problem of minimizing a general matrix, trace function, possibly subject to certain constraints, is approached by means of majorizing this function by one having a simple quadratic shape and whose minimum is easily found. It is shown that the parameter set that minimizes the majorizing function also decreases the matrix trace function, which in turn provides a monotonically convergent algorithm for minimizing the matrix trace function iteratively. Three algorithms based on majorization for solving certain least squares problems are shown to be special cases. In addition, by means of several examples, it is noted how algorithms may be provided for a wide class of statistical optimization tasks for which no satisfactory algorithms seem available.

Key words

trace optimization majorization alternating least squares 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bijleveld, C., & de Leeuw, J. (1987, June).Fitting linear dynamics systems by alternating least squares. Paper presented at the European Meeting of the Psychometric Society, Twente, The Netherlands.Google Scholar
  2. Carroll, J. D., & Chang, J. J. (1972, March).IDIOSCAL: A generalization of INDSCAL allowing IDIOsyncratic reference systems as well as an analytic approximation to INDSCAL. Paper presented at the Spring Meeting of the Psychometric Society, Princeton, N.J.Google Scholar
  3. Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.Google Scholar
  4. de Leeuw, J. (1988). Convergence of the majorization method for Multidimensional Scaling.Journal of Classification, 5, 163–180.Google Scholar
  5. de Leeuw, J., & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland Publishing Company.Google Scholar
  6. de Leeuw, J., Young, F. W., & Takane, Y. (1976). Additive structure in qualitative data: an alternating least squares method with optimal scaling features.Psychometrika, 41, 471–503.Google Scholar
  7. Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank.Psychometrika, 1, 211–218.Google Scholar
  8. Harshman, R. A. (1978, August).Models for analysis of asymmetric relationships among N objects or stimuli. Paper presented at the first joint meeting of the Psychometic Society and the Society for Mathematical Psychology, Hamilton, Ontario.Google Scholar
  9. Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356.Google Scholar
  10. Kiers, H. A. L. (1989). INDSCAL for the analysis of categorical data. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 155–167). Amsterdam: Elsevier Science.Google Scholar
  11. Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane's algorithm for DEDICOM.Psychometrika, 55, 151–158.Google Scholar
  12. Kroonenberg, P. M. (1983).Three mode principal component analysis: Theory and applications. Leiden: DSWO.Google Scholar
  13. Kroonenberg, P. M., & de Leeuw, J. (1980). Principal component analysis of three-mode data by means of alternating least squares algorithms.Psychometrika, 45, 69–97.Google Scholar
  14. Lingoes, J. C., & Borg, I. (1978). A direct approach to individual differences scaling using increasingly complex transformations.Psychometrika, 43, 491–519.Google Scholar
  15. Meulman, J. J. (1986).A distance approach to nonlinear multivariate analysis. Leiden: DSWO.Google Scholar
  16. Ramsay, J. O. (1975). Solving implicit equations in psychometric data analysis.Psychometrika, 40, 337–360.Google Scholar
  17. ten Berge, J. M. F., Knol, D. L., & Kiers, H. A. L. (1988). A treatment of the Orthomax rotation family in terms of diagonalization, and a re-examination of a singular value approach to Varimax rotation.Computational Statistics Quarterly, 3, 207–217.Google Scholar
  18. Wold, H. (1966). Estimation of principal components and related models by iterative least squares. In P. R. Krishnaiah (Ed.),Multivariate analysis II (pp. 391–420). New York: Academic Press.Google Scholar
  19. Young, F. W., de Leeuw, J., & Takane, Y. (1976). Regression with qualitative and quantitative variables: an alternating least squares method with optimal scaling features.Psychometrika, 41, 505–529.Google Scholar

Copyright information

© The Psychometric Society 1990

Authors and Affiliations

  • Henk A. L. Kiers
    • 1
  1. 1.University of GroningenThe Netherlands

Personalised recommendations