Abstract
Dynamic programming methods for matrix permutation problems in combinatorial data analysis can produce globally-optimal solutions for matrices up to size 30×30, but are computationally infeasible for larger matrices because of enormous computer memory requirements. Branch-and-bound methods also guarantee globally-optimal solutions, but computation time considerations generally limit their applicability to matrix sizes no greater than 35×35. Accordingly, a variety of heuristic methods have been proposed for larger matrices, including iterative quadratic assignment, tabu search, simulated annealing, and variable neighborhood search. Although these heuristics can produce exceptional results, they are prone to converge to local optima where the permutation is difficult to dislodge via traditional neighborhood moves (e.g., pairwise interchanges, object-block relocations, object-block reversals, etc.). We show that a heuristic implementation of dynamic programming yields an efficient procedure for escaping local optima. Specifically, we propose applying dynamic programming to reasonably-sized subsequences of consecutive objects in the locally-optimal permutation, identified by simulated annealing, to further improve the value of the objective function. Experimental results are provided for three classic matrix permutation problems in the combinatorial data analysis literature: (a) maximizing a dominance index for an asymmetric proximity matrix; (b) least-squares unidimensional scaling of a symmetric dissimilarity matrix; and (c) approximating an anti-Robinson structure for a symmetric dissimilarity matrix.
Similar content being viewed by others
References
Aarts, E., & Korst, J. (1989). Simulated annealing and Boltzmann machines: A stochastic approach to combinatorial optimization and neural computing. New York: Wiley.
Baker, F.B., & Hubert, L.J. (1977). Applications of combinatorial programming to data analysis: Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 30, 154–164.
Bellman, R. (1962). Dynamic programming treatment of the traveling salesman problem. Journal of the Association for Computing Machinery, 9, 61–63.
Blin, J.M., & Whinston, A.B. (1974). A note on majority rule under transitivity constraints. Management Science, 20, 1439–1440.
Borg, I., & Groenen, P.J.F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York: Springer.
Bowman, V.J., & Colantoni, C.S. (1973). Majority rule under transitivity constraints. Management Science, 19, 1029–1041.
Brusco, M.J. (2001a). Seriation of asymmetric proximity matrices using integer linear programming. British Journal of Mathematical and Statistical Psychology, 54, 367–375.
Brusco, M.J. (2001b). A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. Journal of Classification, 18, 3–33.
Brusco, M.J. (2002). Identifying a reordering of the rows and columns of multiple proximity matrices using multiobjective programming. Journal of Mathematical Psychology, 46, 731–745.
Brusco, M.J. (2006). On the performance of simulated annealing for large-scale L 2 unidimensional scaling. Journal of Classification, 23, 255–268.
Brusco, M.J., & Stahl, S. (2000). Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. Journal of Classification, 17, 197–223.
Brusco, M.J., & Stahl, S. (2001). An interactive approach to multiobjective combinatorial data analysis. Psychometrika, 66, 5–24.
Brusco, M.J., & Stahl, S. (2005a). Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. Psychometrika, 70, 253–270.
Brusco, M.J., & Stahl, S. (2005b). Branch-and-bound applications in combinatorial data analysis. New York: Springer.
Brusco, M.J., & Stahl, S. (2005c). Bicriterion seriation methods for skew-symmetric matrices. British Journal of Mathematical and Statistical Psychology, 58, 333–343.
Brusco, M.J., & Steinley, D. (in press). A comparison of heuristic procedures for minimum within-cluster sums of squares partitioning. Psychometrika
Chenery, H.R., & Watanabe, T. (1958). International comparisons of the structure of production. Econometrica, 26, 487–521.
DeCani, J.S. (1969). Maximum likelihood paired comparison ranking by linear programming. Biometrika, 56, 537–545.
DeCani, J.S. (1972). A branch and bound algorithm for maximum likelihood paired comparison ranking by linear programming. Biometrika, 59, 131–135.
Defays, D. (1978). A short note on a method of seriation. British Journal of Mathematical and Statistical Psychology, 31, 49–53.
de Leeuw, J., & Heiser, W.J. (1977). Convergence of correction-matrix algorithms for multidimensional scaling. In J.C. Lingoes (Ed.), Geometric representations of relational data: Readings in multidimensional scaling (pp. 735–752). Ann Arbor: Mathesis Press.
De Soete, G., Hubert, L., & Arabie, P. (1988). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In E. Diday (Ed.), Data analysis and informatics (Vol. 5, pp. 489–496). Amsterdam: North-Holland.
Flueck, J.A., & Korsh, J.F. (1974). A branch search algorithm for maximum likelihood paired comparison ranking. Biometrika, 61, 621–626.
Fukui, Y. (1986). A more powerful method for triangularizing input-output matrices and the similarity of production structures. Econometrica, 54, 1425–1433.
Garcia, C.G., Pérez-Brito, D., Campos, V., & Marti, R. (2006). Variable neighborhood search for the linear ordering problem. Computers & Operations Research, 33, 3549–3565.
Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness. San Francisco: Freeman.
Glover, F., & Laguna, M. (1993). Tabu search. In C. Reeves (Ed.), Modern heuristic techniques for combinatorial problems (pp. 70–141). Oxford: Blackwell.
Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning. New York: Addison-Wesley.
Groenen, P.J.F. (1993). The majorization approach to multidimensional scaling: Some problems and extensions. Leiden: DSWO Press.
Groenen, P.J.F., & Heiser, W.J. (1996). The tunneling method for global optimization in multidimensional scaling. Psychometrika, 61, 529–550.
Groenen, P.J.F., Heiser, W.J., & Meulman, J.J. (1999). Global optimization in least-squares multidimensional scaling by distance smoothing. Journal of Classification, 16, 225–254.
Grötschel, M., Jünger, M., & Reinelt, G. (1984). A cutting plane algorithm for the linear ordering problem. Operations Research, 32, 1195–1220.
Hansen, P., & Mladenoviĉ, N. (2003). Variable neighborhood search. In F.W. Glover & G.A. Kochenberger (Eds.), Handbook of metaheuristics (pp. 145–184). Norwell: Kluwer Academic.
Held, M., & Karp, R.M. (1962). A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied Mathematics, 10, 196–210.
Holman, E. (1979). Monotonic models for asymmetric proximities. Journal of Mathematical Psychology, 20, 1–15.
Howe, E.C. (1991). A more powerful method for triangularizing input-output matrices: A comment. Econometrica, 59, 521–523.
Hubert, L. (1976). Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 29, 32–52.
Hubert, L., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In J. de Leeuw, W. Heiser, J. Meulman, & F. Critchley (Eds.), Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press.
Hubert, L., & Arabie, P. (1994). The analysis of proximity matrices through sums of matrices having (anti-)Robinson forms. British Journal of Mathematical and Statistical Psychology, 47, 1–40.
Hubert, L., & Arabie, P. (1995). Iterative projection strategies for the least-squares fitting of tree structures to proximity data. British Journal of Mathematical and Statistical Psychology, 48, 281–317.
Hubert, L.J., & Golledge, R.G. (1981). Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation. Psychometrika, 46, 429–441.
Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29, 190–241.
Hubert, L., Arabie, P., & Meulman, J. (1997). Linear and circular unidimensional scaling for symmetric proximity matrices. British Journal of Mathematical and Statistical Psychology, 50, 253–284.
Hubert, L., Arabie, P., & Meulman, J. (1998a). Graph-theoretic representations for proximity matrices through strongly anti-Robinson or circular strongly anti-Robinson matrices. Psychometrika, 63, 341–358.
Hubert, L., Arabie, P., & Meulman, J. (1998b). The representation of symmetric proximity data: Dimensions and classifications. The Computer Journal, 41, 566–577.
Hubert, L., Arabie, P., & Meulman, J. (2001). Combinatorial data analysis: Optimization by dynamic programming. Philadelphia: SIAM.
Hubert, L.J., Arabie, P., & Meulman, J.J. (2002). Linear unidimensional scaling in the L2-Norm: Basic optimization methods using MATLAB. Journal of Classification, 19, 303–328.
Hubert, L., Arabie, P., & Meulman, J. (2006). The structural representation of proximity matrices with MATLAB. Philadelphia: SIAM.
Laguna, M., Marti, R., & Campos, V. (1999). Intensification and diversification with elite tabu search solutions for the linear ordering problem. Computers & Operations Research, 26, 1217–1230.
Lawler, E.L. (1964). A comment on minimum feedback arc sets. IEEE Transactions on Circuit Theory, 11, 296–297.
Murillo, A., Vera, J.F., & Heiser, W.J. (2005). A permutation-translation simulated annealing algorithm for L 1 and L 2 unidimensional scaling. Journal of Classification, 22, 119–138.
Phillips, J.P.N. (1967). A procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 20, 217–225.
Phillips, J.P.N. (1969). A further procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 22, 97–101.
Pliner, V. (1996). Metric unidimensional scaling and global optimization. Journal of Classification, 13, 3–18.
Ranyard, R.H. (1976). An algorithm for maximum likelihood ranking and Slater’s i from paired comparisons. British Journal of Mathematical and Statistical Psychology, 29, 242–248.
Reinelt, G. (1997). LOLIB. http://www.iwr.uni-heidelberg.de/groups/comopt/software/LOLIB
Robinson, W.S. (1951). A method for chronologically ordering archaeological deposits. American Antiquity, 16, 293–301.
Rothkopf, E.Z. (1957). A measure of stimulus similarity and errors in some paired-associate learning tasks. Journal of Experimental Psychology, 53, 94–101.
Schiavinotto, T., & Stützle, T. (2004). The linear ordering problem: Instances, search space analysis and algorithms. Journal of Mathematical Modelling and Algorithms, 3, 367–402.
Shepard, R.N., Kilpatrick, D.W., & Cunningham, J.P. (1975). The internal representation of numbers. Cognitive Psychology, 7, 82–138.
Slater, P. (1961). Inconsistencies in a schedule of paired comparisons. Biometrika, 48, 303–312.
Szczotka, F. (1972). On a method of ordering and clustering of objects. Zastosowania Mathematyki, 13, 23–33.
van Os, B.J. (2000). Dynamic programming for partitioning in multivariate data analysis. Leiden: Leiden University Press.
van Os, B.J., & Meulman, J.J. (2004). Improving dynamic programming strategies for partitioning. Journal of Classification, 21, 207–230.
Author information
Authors and Affiliations
Corresponding author
Additional information
We are extremely grateful to the Associate Editor and two anonymous reviewers for helpful suggestions and corrections.
Rights and permissions
About this article
Cite this article
Brusco, M.J., Köhn, HF. & Stahl, S. Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis. Psychometrika 73, 503–522 (2008). https://doi.org/10.1007/s11336-007-9049-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-007-9049-5