Skip to main content
SpringerLink
Log in

Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis

  • Theory and Methods
  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Dynamic programming methods for matrix permutation problems in combinatorial data analysis can produce globally-optimal solutions for matrices up to size 30×30, but are computationally infeasible for larger matrices because of enormous computer memory requirements. Branch-and-bound methods also guarantee globally-optimal solutions, but computation time considerations generally limit their applicability to matrix sizes no greater than 35×35. Accordingly, a variety of heuristic methods have been proposed for larger matrices, including iterative quadratic assignment, tabu search, simulated annealing, and variable neighborhood search. Although these heuristics can produce exceptional results, they are prone to converge to local optima where the permutation is difficult to dislodge via traditional neighborhood moves (e.g., pairwise interchanges, object-block relocations, object-block reversals, etc.). We show that a heuristic implementation of dynamic programming yields an efficient procedure for escaping local optima. Specifically, we propose applying dynamic programming to reasonably-sized subsequences of consecutive objects in the locally-optimal permutation, identified by simulated annealing, to further improve the value of the objective function. Experimental results are provided for three classic matrix permutation problems in the combinatorial data analysis literature: (a) maximizing a dominance index for an asymmetric proximity matrix; (b) least-squares unidimensional scaling of a symmetric dissimilarity matrix; and (c) approximating an anti-Robinson structure for a symmetric dissimilarity matrix.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aarts, E., & Korst, J. (1989). Simulated annealing and Boltzmann machines: A stochastic approach to combinatorial optimization and neural computing. New York: Wiley.

    Google Scholar 

  • Baker, F.B., & Hubert, L.J. (1977). Applications of combinatorial programming to data analysis: Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 30, 154–164.

    Google Scholar 

  • Bellman, R. (1962). Dynamic programming treatment of the traveling salesman problem. Journal of the Association for Computing Machinery, 9, 61–63.

    Google Scholar 

  • Blin, J.M., & Whinston, A.B. (1974). A note on majority rule under transitivity constraints. Management Science, 20, 1439–1440.

    Article  Google Scholar 

  • Borg, I., & Groenen, P.J.F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York: Springer.

    Google Scholar 

  • Bowman, V.J., & Colantoni, C.S. (1973). Majority rule under transitivity constraints. Management Science, 19, 1029–1041.

    Article  Google Scholar 

  • Brusco, M.J. (2001a). Seriation of asymmetric proximity matrices using integer linear programming. British Journal of Mathematical and Statistical Psychology, 54, 367–375.

    Article  PubMed  Google Scholar 

  • Brusco, M.J. (2001b). A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. Journal of Classification, 18, 3–33.

    Google Scholar 

  • Brusco, M.J. (2002). Identifying a reordering of the rows and columns of multiple proximity matrices using multiobjective programming. Journal of Mathematical Psychology, 46, 731–745.

    Article  Google Scholar 

  • Brusco, M.J. (2006). On the performance of simulated annealing for large-scale L 2 unidimensional scaling. Journal of Classification, 23, 255–268.

    Article  Google Scholar 

  • Brusco, M.J., & Stahl, S. (2000). Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. Journal of Classification, 17, 197–223.

    Article  Google Scholar 

  • Brusco, M.J., & Stahl, S. (2001). An interactive approach to multiobjective combinatorial data analysis. Psychometrika, 66, 5–24.

    Article  Google Scholar 

  • Brusco, M.J., & Stahl, S. (2005a). Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. Psychometrika, 70, 253–270.

    Article  Google Scholar 

  • Brusco, M.J., & Stahl, S. (2005b). Branch-and-bound applications in combinatorial data analysis. New York: Springer.

    Google Scholar 

  • Brusco, M.J., & Stahl, S. (2005c). Bicriterion seriation methods for skew-symmetric matrices. British Journal of Mathematical and Statistical Psychology, 58, 333–343.

    Article  PubMed  Google Scholar 

  • Brusco, M.J., & Steinley, D. (in press). A comparison of heuristic procedures for minimum within-cluster sums of squares partitioning. Psychometrika

  • Chenery, H.R., & Watanabe, T. (1958). International comparisons of the structure of production. Econometrica, 26, 487–521.

    Article  Google Scholar 

  • DeCani, J.S. (1969). Maximum likelihood paired comparison ranking by linear programming. Biometrika, 56, 537–545.

    Article  Google Scholar 

  • DeCani, J.S. (1972). A branch and bound algorithm for maximum likelihood paired comparison ranking by linear programming. Biometrika, 59, 131–135.

    Article  Google Scholar 

  • Defays, D. (1978). A short note on a method of seriation. British Journal of Mathematical and Statistical Psychology, 31, 49–53.

    Google Scholar 

  • de Leeuw, J., & Heiser, W.J. (1977). Convergence of correction-matrix algorithms for multidimensional scaling. In J.C. Lingoes (Ed.), Geometric representations of relational data: Readings in multidimensional scaling (pp. 735–752). Ann Arbor: Mathesis Press.

    Google Scholar 

  • De Soete, G., Hubert, L., & Arabie, P. (1988). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In E. Diday (Ed.), Data analysis and informatics (Vol. 5, pp. 489–496). Amsterdam: North-Holland.

    Google Scholar 

  • Flueck, J.A., & Korsh, J.F. (1974). A branch search algorithm for maximum likelihood paired comparison ranking. Biometrika, 61, 621–626.

    Article  Google Scholar 

  • Fukui, Y. (1986). A more powerful method for triangularizing input-output matrices and the similarity of production structures. Econometrica, 54, 1425–1433.

    Article  Google Scholar 

  • Garcia, C.G., Pérez-Brito, D., Campos, V., & Marti, R. (2006). Variable neighborhood search for the linear ordering problem. Computers & Operations Research, 33, 3549–3565.

    Article  Google Scholar 

  • Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness. San Francisco: Freeman.

    Google Scholar 

  • Glover, F., & Laguna, M. (1993). Tabu search. In C. Reeves (Ed.), Modern heuristic techniques for combinatorial problems (pp. 70–141). Oxford: Blackwell.

    Google Scholar 

  • Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning. New York: Addison-Wesley.

    Google Scholar 

  • Groenen, P.J.F. (1993). The majorization approach to multidimensional scaling: Some problems and extensions. Leiden: DSWO Press.

    Google Scholar 

  • Groenen, P.J.F., & Heiser, W.J. (1996). The tunneling method for global optimization in multidimensional scaling. Psychometrika, 61, 529–550.

    Article  Google Scholar 

  • Groenen, P.J.F., Heiser, W.J., & Meulman, J.J. (1999). Global optimization in least-squares multidimensional scaling by distance smoothing. Journal of Classification, 16, 225–254.

    Article  Google Scholar 

  • Grötschel, M., Jünger, M., & Reinelt, G. (1984). A cutting plane algorithm for the linear ordering problem. Operations Research, 32, 1195–1220.

    Article  Google Scholar 

  • Hansen, P., & Mladenoviĉ, N. (2003). Variable neighborhood search. In F.W. Glover & G.A. Kochenberger (Eds.), Handbook of metaheuristics (pp. 145–184). Norwell: Kluwer Academic.

    Google Scholar 

  • Held, M., & Karp, R.M. (1962). A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied Mathematics, 10, 196–210.

    Article  Google Scholar 

  • Holman, E. (1979). Monotonic models for asymmetric proximities. Journal of Mathematical Psychology, 20, 1–15.

    Article  Google Scholar 

  • Howe, E.C. (1991). A more powerful method for triangularizing input-output matrices: A comment. Econometrica, 59, 521–523.

    Article  Google Scholar 

  • Hubert, L. (1976). Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 29, 32–52.

    Google Scholar 

  • Hubert, L., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In J. de Leeuw, W. Heiser, J. Meulman, & F. Critchley (Eds.), Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press.

    Google Scholar 

  • Hubert, L., & Arabie, P. (1994). The analysis of proximity matrices through sums of matrices having (anti-)Robinson forms. British Journal of Mathematical and Statistical Psychology, 47, 1–40.

    Google Scholar 

  • Hubert, L., & Arabie, P. (1995). Iterative projection strategies for the least-squares fitting of tree structures to proximity data. British Journal of Mathematical and Statistical Psychology, 48, 281–317.

    Google Scholar 

  • Hubert, L.J., & Golledge, R.G. (1981). Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation. Psychometrika, 46, 429–441.

    Article  Google Scholar 

  • Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29, 190–241.

    Google Scholar 

  • Hubert, L., Arabie, P., & Meulman, J. (1997). Linear and circular unidimensional scaling for symmetric proximity matrices. British Journal of Mathematical and Statistical Psychology, 50, 253–284.

    Google Scholar 

  • Hubert, L., Arabie, P., & Meulman, J. (1998a). Graph-theoretic representations for proximity matrices through strongly anti-Robinson or circular strongly anti-Robinson matrices. Psychometrika, 63, 341–358.

    Article  Google Scholar 

  • Hubert, L., Arabie, P., & Meulman, J. (1998b). The representation of symmetric proximity data: Dimensions and classifications. The Computer Journal, 41, 566–577.

    Article  Google Scholar 

  • Hubert, L., Arabie, P., & Meulman, J. (2001). Combinatorial data analysis: Optimization by dynamic programming. Philadelphia: SIAM.

    Google Scholar 

  • Hubert, L.J., Arabie, P., & Meulman, J.J. (2002). Linear unidimensional scaling in the L2-Norm: Basic optimization methods using MATLAB. Journal of Classification, 19, 303–328.

    Article  Google Scholar 

  • Hubert, L., Arabie, P., & Meulman, J. (2006). The structural representation of proximity matrices with MATLAB. Philadelphia: SIAM.

    Google Scholar 

  • Laguna, M., Marti, R., & Campos, V. (1999). Intensification and diversification with elite tabu search solutions for the linear ordering problem. Computers & Operations Research, 26, 1217–1230.

    Article  Google Scholar 

  • Lawler, E.L. (1964). A comment on minimum feedback arc sets. IEEE Transactions on Circuit Theory, 11, 296–297.

    Google Scholar 

  • Murillo, A., Vera, J.F., & Heiser, W.J. (2005). A permutation-translation simulated annealing algorithm for L 1 and L 2 unidimensional scaling. Journal of Classification, 22, 119–138.

    Article  Google Scholar 

  • Phillips, J.P.N. (1967). A procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 20, 217–225.

    Google Scholar 

  • Phillips, J.P.N. (1969). A further procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 22, 97–101.

    Google Scholar 

  • Pliner, V. (1996). Metric unidimensional scaling and global optimization. Journal of Classification, 13, 3–18.

    Article  Google Scholar 

  • Ranyard, R.H. (1976). An algorithm for maximum likelihood ranking and Slater’s i from paired comparisons. British Journal of Mathematical and Statistical Psychology, 29, 242–248.

    Google Scholar 

  • Reinelt, G. (1997). LOLIB. http://www.iwr.uni-heidelberg.de/groups/comopt/software/LOLIB

  • Robinson, W.S. (1951). A method for chronologically ordering archaeological deposits. American Antiquity, 16, 293–301.

    Article  Google Scholar 

  • Rothkopf, E.Z. (1957). A measure of stimulus similarity and errors in some paired-associate learning tasks. Journal of Experimental Psychology, 53, 94–101.

    Article  PubMed  Google Scholar 

  • Schiavinotto, T., & Stützle, T. (2004). The linear ordering problem: Instances, search space analysis and algorithms. Journal of Mathematical Modelling and Algorithms, 3, 367–402.

    Article  Google Scholar 

  • Shepard, R.N., Kilpatrick, D.W., & Cunningham, J.P. (1975). The internal representation of numbers. Cognitive Psychology, 7, 82–138.

    Article  Google Scholar 

  • Slater, P. (1961). Inconsistencies in a schedule of paired comparisons. Biometrika, 48, 303–312.

    Google Scholar 

  • Szczotka, F. (1972). On a method of ordering and clustering of objects. Zastosowania Mathematyki, 13, 23–33.

    Google Scholar 

  • van Os, B.J. (2000). Dynamic programming for partitioning in multivariate data analysis. Leiden: Leiden University Press.

    Google Scholar 

  • van Os, B.J., & Meulman, J.J. (2004). Improving dynamic programming strategies for partitioning. Journal of Classification, 21, 207–230.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Marketing, College of Business, Florida State University, Tallahassee, FL, 32306-1110, USA

    Michael J. Brusco

  2. University of Missouri-Columbia, Columbia, MO, USA

    Hans-Friedrich Köhn

  3. Tallahassee, FL, USA

    Stephanie Stahl

Authors
  1. Michael J. Brusco
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hans-Friedrich Köhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephanie Stahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael J. Brusco.

Additional information

We are extremely grateful to the Associate Editor and two anonymous reviewers for helpful suggestions and corrections.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Brusco, M.J., Köhn, HF. & Stahl, S. Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis. Psychometrika 73, 503–522 (2008). https://doi.org/10.1007/s11336-007-9049-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-007-9049-5

Keywords

Search

Navigation