Psychometrika

, 73:503 | Cite as

Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis

  • Michael J. Brusco
  • Hans-Friedrich Köhn
  • Stephanie Stahl
Theory and Methods

Abstract

Dynamic programming methods for matrix permutation problems in combinatorial data analysis can produce globally-optimal solutions for matrices up to size 30×30, but are computationally infeasible for larger matrices because of enormous computer memory requirements. Branch-and-bound methods also guarantee globally-optimal solutions, but computation time considerations generally limit their applicability to matrix sizes no greater than 35×35. Accordingly, a variety of heuristic methods have been proposed for larger matrices, including iterative quadratic assignment, tabu search, simulated annealing, and variable neighborhood search. Although these heuristics can produce exceptional results, they are prone to converge to local optima where the permutation is difficult to dislodge via traditional neighborhood moves (e.g., pairwise interchanges, object-block relocations, object-block reversals, etc.). We show that a heuristic implementation of dynamic programming yields an efficient procedure for escaping local optima. Specifically, we propose applying dynamic programming to reasonably-sized subsequences of consecutive objects in the locally-optimal permutation, identified by simulated annealing, to further improve the value of the objective function. Experimental results are provided for three classic matrix permutation problems in the combinatorial data analysis literature: (a) maximizing a dominance index for an asymmetric proximity matrix; (b) least-squares unidimensional scaling of a symmetric dissimilarity matrix; and (c) approximating an anti-Robinson structure for a symmetric dissimilarity matrix.

Keywords

Combinatorial data analysis matrix permutation dynamic programming heuristics 

References

  1. Aarts, E., & Korst, J. (1989). Simulated annealing and Boltzmann machines: A stochastic approach to combinatorial optimization and neural computing. New York: Wiley. Google Scholar
  2. Baker, F.B., & Hubert, L.J. (1977). Applications of combinatorial programming to data analysis: Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 30, 154–164. Google Scholar
  3. Bellman, R. (1962). Dynamic programming treatment of the traveling salesman problem. Journal of the Association for Computing Machinery, 9, 61–63. Google Scholar
  4. Blin, J.M., & Whinston, A.B. (1974). A note on majority rule under transitivity constraints. Management Science, 20, 1439–1440. CrossRefGoogle Scholar
  5. Borg, I., & Groenen, P.J.F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York: Springer. Google Scholar
  6. Bowman, V.J., & Colantoni, C.S. (1973). Majority rule under transitivity constraints. Management Science, 19, 1029–1041. CrossRefGoogle Scholar
  7. Brusco, M.J. (2001a). Seriation of asymmetric proximity matrices using integer linear programming. British Journal of Mathematical and Statistical Psychology, 54, 367–375. CrossRefPubMedGoogle Scholar
  8. Brusco, M.J. (2001b). A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. Journal of Classification, 18, 3–33. Google Scholar
  9. Brusco, M.J. (2002). Identifying a reordering of the rows and columns of multiple proximity matrices using multiobjective programming. Journal of Mathematical Psychology, 46, 731–745. CrossRefGoogle Scholar
  10. Brusco, M.J. (2006). On the performance of simulated annealing for large-scale L 2 unidimensional scaling. Journal of Classification, 23, 255–268. CrossRefGoogle Scholar
  11. Brusco, M.J., & Stahl, S. (2000). Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. Journal of Classification, 17, 197–223. CrossRefGoogle Scholar
  12. Brusco, M.J., & Stahl, S. (2001). An interactive approach to multiobjective combinatorial data analysis. Psychometrika, 66, 5–24. CrossRefGoogle Scholar
  13. Brusco, M.J., & Stahl, S. (2005a). Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. Psychometrika, 70, 253–270. CrossRefGoogle Scholar
  14. Brusco, M.J., & Stahl, S. (2005b). Branch-and-bound applications in combinatorial data analysis. New York: Springer. Google Scholar
  15. Brusco, M.J., & Stahl, S. (2005c). Bicriterion seriation methods for skew-symmetric matrices. British Journal of Mathematical and Statistical Psychology, 58, 333–343. CrossRefPubMedGoogle Scholar
  16. Brusco, M.J., & Steinley, D. (in press). A comparison of heuristic procedures for minimum within-cluster sums of squares partitioning. Psychometrika Google Scholar
  17. Chenery, H.R., & Watanabe, T. (1958). International comparisons of the structure of production. Econometrica, 26, 487–521. CrossRefGoogle Scholar
  18. DeCani, J.S. (1969). Maximum likelihood paired comparison ranking by linear programming. Biometrika, 56, 537–545. CrossRefGoogle Scholar
  19. DeCani, J.S. (1972). A branch and bound algorithm for maximum likelihood paired comparison ranking by linear programming. Biometrika, 59, 131–135. CrossRefGoogle Scholar
  20. Defays, D. (1978). A short note on a method of seriation. British Journal of Mathematical and Statistical Psychology, 31, 49–53. Google Scholar
  21. de Leeuw, J., & Heiser, W.J. (1977). Convergence of correction-matrix algorithms for multidimensional scaling. In J.C. Lingoes (Ed.), Geometric representations of relational data: Readings in multidimensional scaling (pp. 735–752). Ann Arbor: Mathesis Press. Google Scholar
  22. De Soete, G., Hubert, L., & Arabie, P. (1988). The comparative performance of simulated annealing on two problems of combinatorial data analysis. In E. Diday (Ed.), Data analysis and informatics (Vol. 5, pp. 489–496). Amsterdam: North-Holland. Google Scholar
  23. Flueck, J.A., & Korsh, J.F. (1974). A branch search algorithm for maximum likelihood paired comparison ranking. Biometrika, 61, 621–626. CrossRefGoogle Scholar
  24. Fukui, Y. (1986). A more powerful method for triangularizing input-output matrices and the similarity of production structures. Econometrica, 54, 1425–1433. CrossRefGoogle Scholar
  25. Garcia, C.G., Pérez-Brito, D., Campos, V., & Marti, R. (2006). Variable neighborhood search for the linear ordering problem. Computers & Operations Research, 33, 3549–3565. CrossRefGoogle Scholar
  26. Garey, M.R., & Johnson, D.S. (1979). Computers and intractability: A guide to the theory of NP-completeness. San Francisco: Freeman. Google Scholar
  27. Glover, F., & Laguna, M. (1993). Tabu search. In C. Reeves (Ed.), Modern heuristic techniques for combinatorial problems (pp. 70–141). Oxford: Blackwell. Google Scholar
  28. Goldberg, D.E. (1989). Genetic algorithms in search, optimization, and machine learning. New York: Addison-Wesley. Google Scholar
  29. Groenen, P.J.F. (1993). The majorization approach to multidimensional scaling: Some problems and extensions. Leiden: DSWO Press. Google Scholar
  30. Groenen, P.J.F., & Heiser, W.J. (1996). The tunneling method for global optimization in multidimensional scaling. Psychometrika, 61, 529–550. CrossRefGoogle Scholar
  31. Groenen, P.J.F., Heiser, W.J., & Meulman, J.J. (1999). Global optimization in least-squares multidimensional scaling by distance smoothing. Journal of Classification, 16, 225–254. CrossRefGoogle Scholar
  32. Grötschel, M., Jünger, M., & Reinelt, G. (1984). A cutting plane algorithm for the linear ordering problem. Operations Research, 32, 1195–1220. CrossRefGoogle Scholar
  33. Hansen, P., & Mladenoviĉ, N. (2003). Variable neighborhood search. In F.W. Glover & G.A. Kochenberger (Eds.), Handbook of metaheuristics (pp. 145–184). Norwell: Kluwer Academic. Google Scholar
  34. Held, M., & Karp, R.M. (1962). A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied Mathematics, 10, 196–210. CrossRefGoogle Scholar
  35. Holman, E. (1979). Monotonic models for asymmetric proximities. Journal of Mathematical Psychology, 20, 1–15. CrossRefGoogle Scholar
  36. Howe, E.C. (1991). A more powerful method for triangularizing input-output matrices: A comment. Econometrica, 59, 521–523. CrossRefGoogle Scholar
  37. Hubert, L. (1976). Seriation using asymmetric proximity measures. British Journal of Mathematical and Statistical Psychology, 29, 32–52. Google Scholar
  38. Hubert, L., & Arabie, P. (1986). Unidimensional scaling and combinatorial optimization. In J. de Leeuw, W. Heiser, J. Meulman, & F. Critchley (Eds.), Multidimensional data analysis (pp. 181–196). Leiden: DSWO Press. Google Scholar
  39. Hubert, L., & Arabie, P. (1994). The analysis of proximity matrices through sums of matrices having (anti-)Robinson forms. British Journal of Mathematical and Statistical Psychology, 47, 1–40. Google Scholar
  40. Hubert, L., & Arabie, P. (1995). Iterative projection strategies for the least-squares fitting of tree structures to proximity data. British Journal of Mathematical and Statistical Psychology, 48, 281–317. Google Scholar
  41. Hubert, L.J., & Golledge, R.G. (1981). Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation. Psychometrika, 46, 429–441. CrossRefGoogle Scholar
  42. Hubert, L., & Schultz, J. (1976). Quadratic assignment as a general data analysis strategy. British Journal of Mathematical and Statistical Psychology, 29, 190–241. Google Scholar
  43. Hubert, L., Arabie, P., & Meulman, J. (1997). Linear and circular unidimensional scaling for symmetric proximity matrices. British Journal of Mathematical and Statistical Psychology, 50, 253–284. Google Scholar
  44. Hubert, L., Arabie, P., & Meulman, J. (1998a). Graph-theoretic representations for proximity matrices through strongly anti-Robinson or circular strongly anti-Robinson matrices. Psychometrika, 63, 341–358. CrossRefGoogle Scholar
  45. Hubert, L., Arabie, P., & Meulman, J. (1998b). The representation of symmetric proximity data: Dimensions and classifications. The Computer Journal, 41, 566–577. CrossRefGoogle Scholar
  46. Hubert, L., Arabie, P., & Meulman, J. (2001). Combinatorial data analysis: Optimization by dynamic programming. Philadelphia: SIAM. Google Scholar
  47. Hubert, L.J., Arabie, P., & Meulman, J.J. (2002). Linear unidimensional scaling in the L2-Norm: Basic optimization methods using MATLAB. Journal of Classification, 19, 303–328. CrossRefGoogle Scholar
  48. Hubert, L., Arabie, P., & Meulman, J. (2006). The structural representation of proximity matrices with MATLAB. Philadelphia: SIAM. Google Scholar
  49. Laguna, M., Marti, R., & Campos, V. (1999). Intensification and diversification with elite tabu search solutions for the linear ordering problem. Computers & Operations Research, 26, 1217–1230. CrossRefGoogle Scholar
  50. Lawler, E.L. (1964). A comment on minimum feedback arc sets. IEEE Transactions on Circuit Theory, 11, 296–297. Google Scholar
  51. Murillo, A., Vera, J.F., & Heiser, W.J. (2005). A permutation-translation simulated annealing algorithm for L 1 and L 2 unidimensional scaling. Journal of Classification, 22, 119–138. CrossRefGoogle Scholar
  52. Phillips, J.P.N. (1967). A procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 20, 217–225. Google Scholar
  53. Phillips, J.P.N. (1969). A further procedure for determining Slater’s i and all nearest adjoining orders. British Journal of Mathematical and Statistical Psychology, 22, 97–101. Google Scholar
  54. Pliner, V. (1996). Metric unidimensional scaling and global optimization. Journal of Classification, 13, 3–18. CrossRefGoogle Scholar
  55. Ranyard, R.H. (1976). An algorithm for maximum likelihood ranking and Slater’s i from paired comparisons. British Journal of Mathematical and Statistical Psychology, 29, 242–248. Google Scholar
  56. Robinson, W.S. (1951). A method for chronologically ordering archaeological deposits. American Antiquity, 16, 293–301. CrossRefGoogle Scholar
  57. Rothkopf, E.Z. (1957). A measure of stimulus similarity and errors in some paired-associate learning tasks. Journal of Experimental Psychology, 53, 94–101. CrossRefPubMedGoogle Scholar
  58. Schiavinotto, T., & Stützle, T. (2004). The linear ordering problem: Instances, search space analysis and algorithms. Journal of Mathematical Modelling and Algorithms, 3, 367–402. CrossRefGoogle Scholar
  59. Shepard, R.N., Kilpatrick, D.W., & Cunningham, J.P. (1975). The internal representation of numbers. Cognitive Psychology, 7, 82–138. CrossRefGoogle Scholar
  60. Slater, P. (1961). Inconsistencies in a schedule of paired comparisons. Biometrika, 48, 303–312. Google Scholar
  61. Szczotka, F. (1972). On a method of ordering and clustering of objects. Zastosowania Mathematyki, 13, 23–33. Google Scholar
  62. van Os, B.J. (2000). Dynamic programming for partitioning in multivariate data analysis. Leiden: Leiden University Press. Google Scholar
  63. van Os, B.J., & Meulman, J.J. (2004). Improving dynamic programming strategies for partitioning. Journal of Classification, 21, 207–230. CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2007

Authors and Affiliations

  • Michael J. Brusco
    • 1
  • Hans-Friedrich Köhn
    • 2
  • Stephanie Stahl
    • 3
  1. 1.Department of Marketing, College of BusinessFlorida State UniversityTallahasseeUSA
  2. 2.University of Missouri-ColumbiaColumbiaUSA
  3. 3.TallahasseeUSA

Personalised recommendations