Abstract
Additive clustering was originally developed within cognitive psychology to enable the development of featural models of human mental representation. The representational flexibility of additive clustering, however, suggests its more general application to modeling complicated relationships between objects in non-psychological domains of interest. This paper describes, demonstrates, and evaluates a simple method for learning additive clustering models, based on the combinatorial optimization approach known as Population-Based Incremental Learning. The performance of this new method is shown to be comparable with previously developed methods over a set of ‘benchmark’ data sets. In addition, the method developed here has the potential, by using a Bayesian analysis of model complexity that relies on an estimate of data precision, to determine the appropriate number of clusters to include in a model.
Article PDF
Similar content being viewed by others
References
Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716-723.
Arabie, P., & Carroll, J. D. (1980). MAPCLUS: A mathematical programming approach to fitting the ADCLUS model. Psychometrika, 45:2, 211-235.
Arabie, P., Carroll, J. D., & DeSarbo, W. S. (1987). Three-way scaling and clustering. Newbury Park, CA: Sage.
Arabie, P., & Shepard, R. N. (1973). Representation of similarities as combinations of discrete, overlapping properties. In Mathematical Psychological Meeting, Montréal.
Baluja, S. (1994). Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Technical Report CMU-CS-94-163, Carnegie-Mellon University.
Baluja, S., & Davies, S. (1997). Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space. Technical Report CS-CMU-97-107, Carnegie-Mellon University.
Baluja, S., & Davies, S. (1998). Fast probabilistic modeling for combinatorial optimization. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (pp. 469-476). Madison, WI: AAAI Press/MIT Press.
Boyan, J. A., & Moore, A. W. (1998). Learning evaluation function for global optimization and boolean satisfiability. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (pp. 3-10). Madison, WI: AAAI Press/MIT Press.
Breiger, R. L., Boorman, S. A., & Arabie, P. (1975). An algorithm for clustering relational data with applications to social network analysis and comparison with multidimensional scaling. Journal of Mathematical Psychology, 12, 328-383.
Chaturvedi, A., & Carroll, J. D. (1994).Analternating combinatorial optimization approach to fitting theINDCLUS and generalized INDCLUS models. Journal of Classification, 11, 155-170.
Cox, T. F., & Cox, M. A. A. (1994). Multidimensional scaling. London: Chapman and Hall.
Ekman, G. (1954). Dimensions of color vision. The Journal of Psychology, 38, 467-474.
Gati, I., & Tversky, A. (1982). Representations of qualitative and quantitative dimensions. Journal of Experimental Psychology: Human Perception and Performance, 8:2, 325-340.
Gregson, R. A. M. (1976). A comparative evaluation of seven similarity models. British Journal of Mathematical and Statistical Psychology, 29, 139-156.
Grünwald, P. (2000). Model selection based on minimum description length. Journal of Mathematical Psychology, 44:1, 133-152.
Hertz, J., Krogh, A., & Palmer, R. G. (1991). Introduction to the theory of neural computing. Redwood City, CA: Addison-Wesley.
Hojo, H. (1982).Amaximumlikelihood method for additive clustering and its applications. Japanese Psychological Research, 25:4, 191-201.
Johnson, E. J., & Tversky, A. (1984). Representations of perceptions of risks. Journal of Experimental Psychology: General, 113:1, 55-70.
Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90:430, 773-795.
Kruschke, J. K. (1993). Human category learning: Implications for backpropagation models. Connection Science, 5, 3-36.
Lawson, C. L., & Hanson, R. J. (1974). Solving least squares problems. Englewood Cliffs, NJ: Prentice-Hall.
Lee, M. D. (1998). Active cognitive representation and learned categorical perception. In 25th Australasian Experimental Psychology Conference, Hobart.
Lee, M. D. (1999a). Algorithms for representing similarity data. Defence Science and Technology Organisation Research Report DSTO-RR-0152.
Lee, M. D. (1999b). An extraction and regularization approach to additive clustering. Journal of Classification, 16:2, 255-281.
Lee, M. D. (2001a). Determining the dimensionality of multidimensional scaling representations for cognitive modeling. Journal of Mathematical Psychology, 45:1, 149-166.
Lee, M. D. (2001b). On the complexity of additive clustering models. Journal of Mathematical Psychology, 45:1, 131-148.
Miller, G. A., & Nicely, P. E. (1955). An analysis of perceptual confusions among some English consonants. Journal of the Acoustical Society of America, 27, 338-352.
Mirkin, B. G. (1987). Additive clustering and qualitative factor analysis methods for similarity matrices. Journal of Classification, 4, 7-31.
Myung, I. J., Balasubramanian, V., & Pitt, M. A. (2000). Counting probability distributions: Differential geometry and model selection. In Proceedings of the National Academy of Sciences, 97, 11170-11175.
Myung, I. J., & Pitt, M. A. (1997). Applying Occam's razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review, 4:1, 79-95.
Raftery, A. E. (1999). Bayes factors and BIC: Comment on Weakliem. Sociological Methods and Research, 27, 411-427.
Roethlisberger, F. J., & Dickson, W. J. (1939). Management and the worker. Cambridge, MA: Harvard University Press.
Rosenberg, S., & Kim, M. P. (1975). The method of sorting as a data-generating procedure in multivariate research. Multivariate Behavioral Research, 10, 489-502.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6:2, 461-464.
Shepard, R. N. (1974). Representation of structure in similarity data: Problems and prospects. Psychometrika, 39:4, 373-422.
Shepard, R. N., & Arabie, P. (1979). Additive clustering representations of similarities as combinations of discrete overlapping properties. Psychological Review, 86:2, 87-123.
Shepard, R. N., Kilpatrick, D.W., & Cunningham, J. P. (1975). The internal representation of numbers. Cognitive Psychology, 7, 82-138.
Tenenbaum, J. B. (1996). Learning the structure of similarity. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Advances in neural information processing systems (Vol. 2). Cambridge, MA: MIT Press.
Woodruff, C. J. (1998). Establishing a psychophysics of texture. In 25th Australasian Experimental Psychology Conference, Hobart.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Lee, M.D. A Simple Method for Generating Additive Clustering Models with Limited Complexity. Machine Learning 49, 39–58 (2002). https://doi.org/10.1023/A:1014112506867
Issue Date:
DOI: https://doi.org/10.1023/A:1014112506867