Advertisement

Psychometrika

, Volume 61, Issue 3, pp 529–550 | Cite as

The tunneling method for global optimization in multidimensional scaling

  • Patrick J. F. Groenen
  • Willem J. Heiser
Article

Abstract

This paper focuses on the problem of local minima of the STRESS function. It turns out that unidimensional scaling is particularly prone to local minima, whereas full dimensional scaling with Euclidean distances has a local minimum that is global. For intermediate dimensionality with Euclidean distances it depends on the dissimilarities how severe the local minimum problem is. For city-block distances in any dimensionality many different local minima are found. A simulation experiment is presented that indicates under what conditions local minima can be expected. We introduce the tunneling method for global minimization, and adjust it for multidimensional scaling with general Minkowski distances. The tunneling method alternates a local search step, in which a local minimum is sought, with a tunneling step in which a different configuration is sought with the same STRESS as the previous local minimum. In this manner successively better local minima are obtained, and experimentation so far shows that the last one is often a global minimum.

Key words

multidimensional scaling iterative majorization global optimization tunneling method 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arabie, P. (1991). Was Euclid an unnecessarily sophisticated psychologist?Psychometrika, 56, 567–587.Google Scholar
  2. Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675.Google Scholar
  3. Commandeur, J. J. F. (1992).Missing data in the distance approach to Principal Component Analysis (Research Rep. No. RR-92-07). Leiden: Department of Data Theory.Google Scholar
  4. Critchley, F. (1986). Dimensionality theorems in MDS and HCA. In E. Diday et al. (Eds),Data analysis and informatics, Vol. 4 (pp. 45–70). Amsterdam: North-Holland.Google Scholar
  5. Defays, D. (1978). A short note on a method of seriation.British Journal of Mathematical and Statistical Psychology, 3, 49–53.Google Scholar
  6. de Leeuw, J. (1977). Applications of convex analysis to multidimensional scaling. In J. R. Barra, F. Brodeau, G. Romier, & B. van Cutsem (Eds.),Recent development in statistics (pp. 133–145). Amsterdam: North-Holland.Google Scholar
  7. de Leeuw, J. (1988). Convergence of the majorization method for multidimensional scaling.Journal of Classification, 5, 163–180.Google Scholar
  8. de Leeuw, J. (1993).Fitting distances by least squares. Unpublished manuscript.Google Scholar
  9. de Leeuw, J., & Heiser, W. J. (1977). Convergence of correction matrix algorithms for multidimensional scaling. In J. C. Lingoes, E. Roskam, & I. Borg (Eds.),Geometric representations of relational data (pp. 735–752). Ann Arbor: Mathesis Press.Google Scholar
  10. de Leeuw, J. & Heiser, W. J. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis, Vol. V (pp. 501–522). Amsterdam: North-Holland.Google Scholar
  11. De Soete, G., Hubert, L., & Arabie, P. (1988). On the use of simulated annealing for combinatorial data analysis. In W. Gaul & M. Schader (Eds.),Data, expert, knowledge and decisions (pp. 329–340). Berlin: Springer-Verlag.Google Scholar
  12. Dinkelbach, W. (1967). On nonlinear fractional programming.Management Science, 13, 492–498.Google Scholar
  13. Funk, S. G., Horowitz, A. D., Lipshitz, R., & Young, F. W. (1974). The perceived structure of American ethnic groups: The use of multidimensional scaling in stereotype research.Personality and Social Psychology Bulletin, 1, 66–68.Google Scholar
  14. Gomez, S., & Levy, A. V. (1982). The tunneling method for solving the constrained global optimization problem with non-connected feasible regions. In J. P. Hennart (Ed.),Lecture notes in mathematics, 909 (pp. 34–47). Berlin: Springer-Verlag.Google Scholar
  15. Green, P. E., Carmone, F. J. Jr., & Smith, S. M. (1989).Multidimensional scaling, concepts and applications. Boston: Allyn and Bacon.Google Scholar
  16. Groenen, P. J. F. (1993).The majorization approach to multidimensional scaling: Some problems and extensions. Leiden: DSWO Press.Google Scholar
  17. Groenen, P. J. F., & Heiser, W. J. (1991).An improved tunneling function for finding a decreasing series of local minima (Research Rep. No. RR-91-06). Leiden: Department of Data Theory.Google Scholar
  18. Groenen, P. J. F., de Leeuw, J., & Mathar, R. (1996). Least squares multidimensional scaling with transformed distances. In W. Gaul & D. Pfeifer (Eds.),Studies in classification, data analysis, and knowledge organization (pp. 177–185). Berlin: Springer.Google Scholar
  19. Groenen, P. J. F., Mathar, R., & Heiser, W. J. (1995). The majorization approach to multidimensional scaling for Minkowski distances.Journal of Classification, 12, 3–19.Google Scholar
  20. Hardy, G. H., Littlewood, J. E. & Pólya, G. (1952).Inequalities (2nd ed.). Cambridge: University Press.Google Scholar
  21. Heiser, W. J. (1989). The city-block model for three-way multidimensional scaling. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 395–404). Amsterdam: North-Holland.Google Scholar
  22. Heiser, W. J. (1991). A generalized majorization method for least squares multidimensional scaling of pseudodistances that may be negative.Psychometrika, 56, 7–27.Google Scholar
  23. Heiser, W. J. (1995).Convergent computation by iterative majorization: Theory and applications in multidimensional data analysis. In W. J. Krzanowski (Eds.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.Google Scholar
  24. Heiser, W. J., & de Leeuw, J. (1977).How to use SMACOF-1 (Research Rep. No. UG-86-02). Leiden: Department of Data Theory.Google Scholar
  25. Heiser, W. J., & Groenen, P. J. F. (1994).Cluster differences scaling with a within clusters loss component and a fuzzy successive approximation strategy to avoid local minima (Research Rep. No. RR-94-03). Leiden: Department of Data Theory.Google Scholar
  26. Hubert, L. J., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In J. de Leeuw, W. J. Heiser, J. Meulman & F. Critchley (Eds.),Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press.Google Scholar
  27. Hubert, L. J., Arabie, P., & Hesson-McInnis, M. (1992). Multidimensional scaling in the city-block metric: A combinatorial approach.Journal of Classification, 9, 211–236.Google Scholar
  28. Kruskal, J. B. (1964a). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis.Psychometrika, 29, 1–28.Google Scholar
  29. Kruskal, J. B. (1964b). Nonmetric multidimensional scaling: A numerical method.Psychometrika, 29, 115–129.Google Scholar
  30. Kruskal, J. B., Young, F. W., & Seery, J. B. (1977).How to use KYST-2, a very flexible program to do multidimensional scaling and unfolding. Murray Hill, NJ: AT&T Bell Laboratories.Google Scholar
  31. Levy, A. V., & Gomez, S. (1985). The tunneling method applied to global optimization. In P. T. Boggs, R. H. Byrd, & R. B. Schnabel (Eds.),Numerical optimization 1984 (pp. 213–244). Philadelphia: SIAM.Google Scholar
  32. Mathar, R., & Groenen, P. J. F. (1991). Algorithms in convex analysis applied to multidimensional scaling. In E. Diday & Y. Lechevallier (Eds.),Symbolic-numeric data analysis and learning (pp. 45–56). Commack, NY: Nova Science Publishers.Google Scholar
  33. Meulman, J. J. (1986).A distance approach to nonlinear multivariate analysis. Leiden: DSWO Press.Google Scholar
  34. Meulman, J. J. (1992). The integration of multidimensional scaling and multivariate analysis with optimal transformations.Psychometrika, 57, 539–565.Google Scholar
  35. Montalvo, A. (1979).Development of a new algorithm for the global minimization of functions Unpublished doctoral dissertation, Universidad Nacional Autonoma de Mexico.Google Scholar
  36. Robinson, W. S. (1951). A method for chronologically ordering archaeological deposits.American Antiquity, 16, 293–301.Google Scholar
  37. Shepard, R. N. (1962). Analysis of proximities: Multidimensional scaling with an unknown distance function.Psychometrika, 27, 125–140.Google Scholar
  38. Tijssen, R. J. W. (1992).Cartography of science: Scientometric mapping with multidimensional scaling methods. Leiden: DSWO Press.Google Scholar
  39. Torgerson, W. S. (1958).Theory and methods of scaling. New York: Wiley.Google Scholar
  40. Tucker, W. S. (1951).A method for synthesis of factor analysis studies (Personel Research Section Rep. No. 984). Washington DC: Department of the Army.Google Scholar
  41. Vilkov, A. V., Zhidkov, N. P., & Shchedrin, B. M. (1975). A method of finding the global minimum of a function of one variable.USSR Computational Mathematics and Mathematical Physics, 15, 1040–1042.Google Scholar
  42. Wagenaar, W. A., & Padmos, P. (1971). Quantitative interpretation of stress in Kruskal's multidimensional scaling technique.British Journal of Mathematical and Statistical Psychology, 24, 101–110.Google Scholar
  43. Weeks, D. G., & Bentler, P. M. (1982). Restricted multidimensional scaling models for asymmetric proximities.Psychometrika, 47, 201–208.Google Scholar
  44. Zielman, B. (1991).Three-way scaling of asymmetric proximities (Research Rep. No. RR-91-01). Leiden: Department of Data Theory.Google Scholar

Copyright information

© The Psychometric Society 1996

Authors and Affiliations

  • Patrick J. F. Groenen
    • 1
  • Willem J. Heiser
    • 1
  1. 1.Department of Data Theory, Faculty of Social and Behavioural SciencesLeiden UniversityThe Netherlands

Personalised recommendations