Abstract
Hierarchical classes models are quasi-order retaining Boolean decomposition models for N-way N-mode binary data. To fit these models to data, rationally started alternating least squares (or, equivalently, alternating least absolute deviations) algorithms have been proposed. Extensive simulation studies showed that these algorithms succeed quite well in recovering the underlying truth but frequently end in a local minimum. In this paper we evaluate whether or not this local minimum problem can be mitigated by means of two common strategies for avoiding local minima in combinatorial data analysis: simulated annealing (SA) and use of a multistart procedure. In particular, we propose a generic SA algorithm for hierarchical classes analysis and three different types of random starts. The effectiveness of the SA algorithm and the random starts is evaluated by reanalyzing data sets of previous simulation studies. The reported results support the use of the proposed SA algorithm in combination with a random multistart procedure, regardless of the properties of the data set under study.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts, E.H., & Lenstra, J.K. (1997). Local search in combinatorial optimization. Chichester, UK: Wiley.
Al-Sultan, K.S., & Khan, M.M. (1996). Computational experience on four algorithms for the hard clustering problem. Pattern Recognition Letters, 17, 295–308.
Brusco, M.J. (2001). A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. Journal of Classification, 18, 3–13.
Brusco, M.J., & Stahl, S. (2000). Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. Journal of Classification, 17, 197–223.
Ceulemans, E., & Van Mechelen, I. (2004). Tucker2 hierarchical classes analysis. Psychometrika, 69, 375–399.
Ceulemans, E., & Van Mechelen, I. (2005). Hierarchical classes models for three-way three-mode binary data: Interrelations and model selection. Psychometrika, 70, 1–10.
Ceulemans, E., Van Mechelen, I., & Leenen, I. (2003). Tucker3 hierarchical classes analysis. Psychometrika, 68, 413–433.
De Boeck, P., & Rosenberg, S. (1988). Hierarchical classes: Model and data analysis. Psychometrika, 53, 361–381.
Gara, M., & Rosenberg, S. (1990). A set-theoretical model of person perception. Behavioral Research, 25, 275–293.
Hand, D.J., & Krzanowski, W.J. (2005). Optimising k-means clustering results with standard software packages. Computational Statistics and Data Analysis, 49, 969–973.
Hubert, L., Arabie, P., & Hesson-McInnis, M. (1992). Multidimensional-scaling in the city-block metric—a combinatorial approach. Journal of Classification, 9, 211–236.
Klein, R.W., & Dubes, R.C. (1989). Experiments in projection and clustering by simulated annealing. Pattern Recognition, 22, 213–220.
Kuppens, P., Van Mechelen, I., Smits, D.J.M., De Boeck, P., & Ceulemans, E. (2007). Individual differences in patterns of appraisal and anger experience. Cognition & Emotion, 21, 689–713.
Leenen, I., & Van Mechelen, I. (1998). A branch-and-bound algorithm for Boolean regression. In I. Balderjahn, R. Mathar, & M. Schader, Data highways and information flooding, A challenge for classification and data analysis (pp. 164–171). Berlin: Springer.
Leenen, I., & Van Mechelen, I. (2001). An evaluation of two algorithms for hierarchical classes analysis. Journal of Classification, 18, 57–60.
Leenen, I., Van Mechelen, I., De Boeck, P., & Rosenberg, S. (1999). indclas: A three-way hierarchical classes model. Psychometrika, 64, 9–14.
Milligan, G.W. (1980). An examination of the effect of six types of error perturbation on fifteen clustering algorithms. Psychometrika, 45, 325–342.
Murillo, A., Vera, J.F., & Heiser, W.J. (2005). A permutation-translation simulated annealing algorithm for l 1 and l 2 unidimensional scaling. Journal of Classification, 22, 119–138.
Selim, S.Z., & Ismail, M.A. (1984). K-means-type algorithms—A generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 81–87.
Steinley, D. (2003). Local optima in k-means clustering: What you don’t know may hurt you. Psychological Methods, 8, 294–304.
Van Mechelen, I., & De Boeck, P. (1989). Implicit taxonomy in psychiatric diagnosis: A case study. Journal of Social and Clinical Psychology, 8, 276–287.
Van Mechelen, I., & Van Damme, G. (1994). A latent criteria model for choice data. Acta Psychologica, 87, 85–94.
Van Mechelen, I., De Boeck, P., & Rosenberg, S. (1995). The conjunctive model of hierarchical classes. Psychometrika, 60, 505–521.
Vansteelandt, K., & Van Mechelen, I. (1998). Individual differences in situation-behavior profiles: A triple typology model. Journal of Personality and Social Psychology, 75, 751–765.
Vansteelandt, K., & Van Mechelen, I. (2006). Individual differences in anger and sadness: In pursuit of active situational features and psychological processes. Journal of Personality, 74, 871–909.
Author information
Authors and Affiliations
Corresponding author
Additional information
Eva Ceulemans is a post-doctoral fellow of the Fund for Scientific Research Flanders (Belgium). Iwin Leenen is a post-doctoral researcher of the Spanish Ministerio de Educación y Ciencia (programa Ramón y Cajal). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/05/04).
Rights and permissions
About this article
Cite this article
Ceulemans, E., Van Mechelen, I. & Leenen, I. The Local Minima Problem in Hierarchical Classes Analysis: An Evaluation of a Simulated Annealing Algorithm and Various Multistart Procedures. Psychometrika 72, 377–391 (2007). https://doi.org/10.1007/s11336-007-9000-9
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-007-9000-9