Abstract
In many regression applications, users are often faced with difficulties due to nonlinear relationships, heterogeneous subjects, or time series which are best represented by splines. In such applications, two or more regression functions are often necessary to best summarize the underlying structure of the data. Unfortunately, in most cases, it is not known a priori which subset of observations should be approximated with which specific regression function. This paper presents a methodology which simultaneously clusters observations into a preset number of groups and estimates the corresponding regression functions' coefficients, all to optimize a common objective function. We describe the problem and discuss related procedures. A new simulated annealing-based methodology is described as well as program options to accommodate overlapping or nonoverlapping clustering, replications per subject, univariate or multivariate dependent variables, and constraints imposed on cluster membership. Extensive Monte Carlo analyses are reported which investigate the overall performance of the methodology. A consumer psychology application is provided concerning a conjoint analysis investigation of consumer satisfaction determinants. Finally, other applications and extensions of the methodology are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts, E., & Korst, J. (1989).Simulated annealing and Boltzmann machines. New York: Wiley.
Addelman, S. (1962). Orthogonal main effects plans for asymmetrical factorial experiments.Technometrics, 4, 21–46.
Bohachevsky, I. O., Johnson, M. E., & Stein, M. L. (1986). Generalized simulated annealing for function optimization.Technometrics, 28, 209–217.
Carroll, J. D. (1972). Individual differences and multidimensional scaling. In R. N. Shepard, A. K. Romney, & S. Nerlove (Eds.),Multidimensional scaling: Theory and applications in the behavioral sciences (Vol. I, pp. 105–155). New York: Seminar Press.
Davis, L. (1987).Genetic algorithms and simulated annealing. London: Pitman.
DeSarbo, W. S. (1982). GENNCLUS: New models for general nonhierarchical clustering analysis.Psychometrika, 47, 446–469.
DeSarbo, W. S., & Carroll, J. D. (1985). Three-way metric unfolding via weighted least squares.Psychometrika, 50, 275–300.
DeSarbo, W. S., Carroll, J. D., Clark, L. A., & Green, P. E. (1984). Synthesized clustering: A method for amalgamating alternative clustering bases with differential weighting of variables.Psychometrika, 49, 57–78.
DeSarbo, W. S., & Cron, W. L. (1988). A maximum likelihood methodology for clusterwise linear regression.Journal of Classification, 5, 249–282.
DeSarbo, W. S., Mahajan, V. (1984). constrained classification: The use of a priori information in cluster analysis.Psychometrika, 49, 187–215.
DeSarbo, W. S., Oliver, R. L., & De Soete, G. (1986). A probabilistic multidimensional scaling vector model.Applied Psychological Measurement, 10, 79–98.
De Soete, G., DeSarbo, W. S., & Carroll, J. D. (1985). Optimal variable weighting for hierarchical clustering: An alternating least squares algorithm.Journal of Classification, 2/3, 173–192.
De Soete, G., DeSarbo, W. S., Furnas, G. W., & Carroll, J. D. (1984). The presentation of nonsymmetric rectangular proximity data by ultrametric and path length tree structures.Psychometrika, 49, 289–310.
De Soete, G., Hubert, L., & Arabie, P. (1988a). On the use of simulated annealing for combinatorial data analysis. In W. Gaul & M. Schader (Eds.),Data, expert knowledge, and decisions (pp. 328–340). Berlin: Springer Verlag.
De Soete, G., Hubert, L., & Arabie, P. (1988b). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In E. Diday (Ed.),Data analysis and informatics, V (pp. 489–496). Amsterdam: North-Holland.
Dubes, R. C., & Klein, R. (1987, June).Simulated annealing in data analysis. Handout at a talk given at the 1987 Annual Meeting of the Psychometric Society, Montreal, Canada.
Gabor, A., & Granger, D. W. (1966). On the price consciousness of consumers.Applied Statistics, 10, 170–181.
Gidas, B. (1985). Nonstationary Markov chains and convergence of the annealing algorithm.Journal of Statistical Physics, 39, 73–131.
Goldberg, D. E. (1989).Genetic algorithms in search, optimization, and macine learning. Reading: Addison-Wesley.
Green, P. E., & Rao, V. R. (1971). Conjoint measurement for quantifying judgmental data.Journal of Marketing Research, 8, 355–363.
Green, P. E., & Srinivasan, V. (1978). Conjoint analysis in consumer research: Issues and outlook.Journal of Consumer Research, 5, 103–123.
Haggerty, M. R. (1985). Improving the predictive power of conjoint analysis: The use of factor analysis and cluster analysis.Journal of Marketing Research, 22, 168–184.
Henderson, J. M., & Quandt, R. E. (1982).Microeconomic theory: A mathematical approach (3rd ed.). New York: McGraw Hill.
Johnson, M. E. (1988).Simulated annealing and optimization. Syracuse: American Science Press.
Judge, G. G., Griffiths, W. E., Hill, R. C., Lükepohl, H., & Lee, T. (1985).The theory and practice of econometrics. New York: Wiley.
Kamakura, A. W. (1988). A least-squares procedure for benefit segmentation with conjoint experiments.Journal of Marketing Research, 25, 157–167.
Kirkpatrick, S., Gelatt, C. D., & Vechhi, M. P. (1983). Optimization by simulated annealing.Science, 220, 671–680.
Levy, A. V., & Montalvo, A. (1985). The tunneling algorithm for the global minimization of functions.SIAM Journal of Scientific and Statistical Computing, 6, 15–29.
Lin, S., & Kernigham, B. (1973). An effective heuristic algorithm for the traveling salesman problem.Operations Research, 21, 498–516.
Locke, E. A. (1967). Relationship of goal level to performance level.Psychological Reports, 20, 1068.
Locke, E. A. (1982). Relation of goal level to performance with a short work period and multiple goal levels.Journal of Applied Psychology, 67, 512–514.
Locke, E. A., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal setting and task performance: 1969–1980.Psychological Bulletin, 90, 125–152.
Lundy, M. (1986). Applications of the annealing algorithm to combinatorial problems in statistics.Biometrika, 72, 191–198.
MacQueen, J. (1967). Some methods for classification and analysis of multivariate observations.5th Berkeley Symposium on Mathematics, Statistics and Probability, 1, 281–298.
Maddala, G. S. (1976).Econometrics. New York: McGraw-Hill.
Meier, J. (1987). A fast algorithm for clusterwise linear absolute deviations regression.OR Spektrum, 9, 187–189.
Mitra, D., Romeo, F., & Sangiovanni-Vincentelli, A. (in press). Convergence and finite-time behavior of simulated annealing.Advances in Applied Probability.
Ogawa, K. (1987). An approach to simultaneous estimation and segmentation in conjoint analysis.Marketing Science, 6, 66–81.
Oliver, R. L. (1980). A cognitive model of the antecedents and consequences of satisfaction decisions.Journal of Marketing Research, 17, 460–469.
Oliver, R. L., & DeSarbo, W. S. (1988). Response determinants in satisfaction judgments.Journal of Consumer Research, 14, 495–507.
Oliver, R. L., & Swan, J. E. (in press). Consumer perceptions of interpersonal equity and satisfaction in transactions: A field survey approach.Journal of Marketing.
Quandt, R. E. (1972). A new approach to estimating switching regressions.Journal of the American Statistical Association, 67, 306–310.
Sowter, A. P., Gabor, A., & Granger, G. W. (1971). The effect of price on choice.Applied Economics, 3, 167–181.
Späth, H. (1979). Algorithm 39: Clusterwise linear regression.Computing, 22, 367–373.
Späth, H. (1981). Correction to Algorithm 39: Clusterwise linear regression.Computing, 26, 275.
Späth, H. (1982). Algorithm 48: A fast algorithm for clusterwise linear regression.Computing, 29, 175–181.
Späth, H. (1985).Cluster dissection and analysis. New York: Wiley.
Späth, H. (1986a). Clusterwise linear least absolute deviations regression.Computing, 37, 371–378.
Späth, H. (1986b). Clusterwise linear least squares versus least absolute deviations regression: A numerical comparison for a case study. In W. Gaul & M. Schader (Eds.),Classification as a tool of research (pp. 413–422), New York: North Holland.
Späth, H. (1987). Mathematische software zur linearen regression [Mathematical software for linear regression]. Munich: R. Oldenbourg.
van Laarhoven, P. J. M., & Aarts, E. H. L. (1987).Simulated annealing: Theory and applications. Boston: D. Reidel.
Author information
Authors and Affiliations
Additional information
The authors wish to thank the editor, associate editor, and three anonymous reviewers for their insightful comments and thorough review of this manuscript.
Rights and permissions
About this article
Cite this article
DeSarbo, W.S., Oliver, R.L. & Rangaswamy, A. A simulated annealing methodology for clusterwise linear regression. Psychometrika 54, 707–736 (1989). https://doi.org/10.1007/BF02296405
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02296405