Annals of Operations Research

, Volume 131, Issue 1–4, pp 325–372 | Cite as

Multilevel Refinement for Combinatorial Optimisation Problems

  • Chris Walshaw


We consider the multilevel paradigm and its potential to aid the solution of combinatorial optimisation problems. The multilevel paradigm is a simple one, which involves recursive coarsening to create a hierarchy of approximations to the original problem. An initial solution is found (sometimes for the original problem, sometimes the coarsest) and then iteratively refined at each level. As a general solution strategy, the multilevel paradigm has been in use for many years and has been applied to many problem areas (most notably in the form of multigrid techniques). However, with the exception of the graph partitioning problem, multilevel techniques have not been widely applied to combinatorial optimisation problems. In this paper we address the issue of multilevel refinement for such problems and, with the aid of examples and results in graph partitioning, graph colouring and the travelling salesman problem, make a case for its use as a metaheuristic. The results provide compelling evidence that, although the multilevel framework cannot be considered as a panacea for combinatorial problems, it can provide an extremely useful addition to the combinatorial optimisation toolkit. We also give a possible explanation for the underlying process and extract some generic guidelines for its future use on other combinatorial problems.

multilevel refinement combinatorial optimisation metaheuristic graph partitioning travelling salesman graph colouring 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Applegate, D., R. Bixby, V. Chvátal, and W.J. Cook. (1999). “Finding Tours in the TSP.” Technical Report TR99-05, Dept. Comput. Appl. Math., Rice University, Houston, TX.Google Scholar
  2. Barnard, S.T. and H.D. Simon. (1994). “A Fast Multilevel Implementation of Recursive Spectral Bisection for Partitioning Unstructured Problems.” Concurrency: Practice and Experience 6(2), 101–117.Google Scholar
  3. Battiti, R., A. Bertossi, and A. Cappelletti. (1999). “Multilevel Reactive Tabu Search for Graph Partitioning.” Preprint UTM 554, Dip. Mat., University Trento, Italy.Google Scholar
  4. Boman, E.G. and B. Hendrickson. (1996). “A Multilevel Algorithm for Reducing the Envelope of Sparse Matrices.” Technical Report 96-14, SCCM, Stanford University, CA.Google Scholar
  5. Brandt, A. (1988). “Multilevel Computations: Review and Recent Developments.” In S.F. McCormick (ed.), Multigrid Methods: Theory, Applications, and Supercomputing, Proc. of 3rd Copper Mountain Conf. Multigrid Methods, Lecture Notes in Pure and Applied Mathematics, Vol. 110. New York: Marcel Dekker, pp. 35–62.Google Scholar
  6. Bui, T.N. and C. Jones. (1993). “A Heuristic for Reducing Fill-In in Sparse Matrix Factorization.” In R.F. Sincovec et al. (eds.), Parallel Processing for Scientific Computing. Philadelphia, PA: SIAM, pp. 445–452.Google Scholar
  7. Christofides, N. (1975). Graph Theory, an Algorithmic Approach. London: Academic Press.Google Scholar
  8. Cook, W.J. and A. Rohe. (1999). “Computing Minimum-Weight Perfect Matchings.” INFORMS J. Comput. 11(2), 138–148.Google Scholar
  9. Croes, G.A. (1958). “A Method for Solving Traveling Salesman Problems.” Oper. Res. 6, 791–812.Google Scholar
  10. Culberson, J.C., A. Beacham, and D. Papp. (1995). “Hiding Our Colors.” In CP'95 Workshop on Studying and Solving Really Hard Problems, September, pp. 31–42.Google Scholar
  11. Culberson, J.C. and F. Luo. (1996). “Exploring the k-Colorable Landscape with Iterated Greedy.” In D.S. Johnson and M.A. Trick (eds.), Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS, pp. 245–284.Google Scholar
  12. Diekmann, R., R. Luling, B. Monien, and C. Spraner. (1996). “Combining Helpful Sets and Parallel Simulated Annealing for the Graph-Partitioning Problem.” Parallel Algorithms Appl. 8, 61–84.Google Scholar
  13. Farhat, C. (1988). “A Simple and Efficient Automatic FEM Domain Decomposer.” Comput. and Structures 28(5), 579–602.Google Scholar
  14. Fiduccia, C.M. and R.M. Mattheyses. (1982). “A Linear Time Heuristic for Improving Network Partitions.” In Proc. 19th IEEE Design Automation Conf. Piscataway, NJ: IEEE, pp. 175–181.Google Scholar
  15. Fleurent, C. and J.A. Ferland. (1996). “Object-Oriented Implementation of Heuristic Search Methods for Graph Coloring, Maximum Clique and Satisfiability.” In D.S. Johnson and M.A. Trick (eds.), Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS, pp. 619–652.Google Scholar
  16. Garey, M.R. and D.S. Johnson. (1979). Computers and Intractability: A Guide to the Theory of NPCompleteness. San Francisco: Freeman.Google Scholar
  17. Gilbert, J.R., G.L. Miller, and S.-H. Teng. (1998). “Geometric Mesh Partitioning: Implementation and Experiments.” SIAM J. Sci. Comput. 19(6), 2091–2110.CrossRefGoogle Scholar
  18. Glover, F. (1986). “Future Paths for Integer Programming and Links to Artificial Intelligence.” Comput. Oper. Res. 13, 533–549.Google Scholar
  19. Glover, F., M. Parker, and J. Ryan. (1996). “Coloring by Tabu Branch and Bound.” In D.S. Johnson and M.A. Trick (eds.), Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS, pp. 285–307.Google Scholar
  20. Gu, J. and X. Huang. (1994). “Efficient Local Search With Search Space Smoothing: A Case Study of the Traveling Salesman Problem (TSP).” IEEE Trans. Syst. Man and Cybernetics 24(5), 728–735.Google Scholar
  21. Held, M. and R.M. Karp. (1970). “The Traveling Salesman Problem and Minimum Spanning Trees.” Oper. Res. 18, 1138–1162.Google Scholar
  22. Helsgaun, K. (2000). “An Effective Implementation of the Lin-Kernighan Traveling Salesman Heuristic.” Eur. J. Oper. Res. 126, 106–130.CrossRefGoogle Scholar
  23. Hendrickson, B. and T.G. Kolda. (2000). “Graph Partitioning Models for Parallel Computing.” Parallel Comput. 26(12), 1519–1534.CrossRefGoogle Scholar
  24. Hendrickson, B. and R. Leland. (1995a). “A Multilevel Algorithm for Partitioning Graphs.” In S. Karin (ed.), Proc. Supercomputing '95, San Diego. New York: ACM Press.Google Scholar
  25. Hendrickson, B. and R. Leland. (1995b). “The Chaco User's Guide: Version 2.0.” Technical Report SAND 94-2692, Sandia Natl. Lab., Albuquerque, NM, July.Google Scholar
  26. Hertz, A. and D. de Werra. (1987). “Using Tabu Search Techniques for Graph Coloring.” Computing 39, 345–351.CrossRefGoogle Scholar
  27. Hu, Y.F. and J.A. Scott. (2000). “Multilevel Algorithms for Wavefront Reduction.” RAL-TR-2000-031, Comput. Sci. and Engrg. Dept., Rutherford Appleton Lab., Didcot, UK.Google Scholar
  28. Johnson, D.S., C.R. Aragon, L.A. McGeoch, and C. Schevon. (1991). “Optimization by Simulated Annealing: Part II, Graph Coloring and Number Partitioning.” Oper. Res. 39(3), 378–406.Google Scholar
  29. Johnson, D.S. and L.A. McGeoch. (1997). “The Travelling Salesman Problem: A Case Study.” In E. Aarts and J.K. Lenstra (eds.), Local Search in Combinatorial Optimization. Chichester: Wiley, pp. 215–310.Google Scholar
  30. Johnson, D.S. and L.A. McGeoch. (2002). “Experimental Analysis of Heuristics for the STSP.” In The Travelling Salesman Problem and its Variations. Dordrecht: Kluwer Academic, pp. 369–443.Google Scholar
  31. Johnson, D.S. and M.A. Trick (eds.). (1996). Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS.Google Scholar
  32. Joslin, D.E. and D.P. Clements. (1999). " "Squeaky Wheel" Optimization.” J. Artificial Intelligence Res.10, 353–373.Google Scholar
  33. Karypis, G. and V. Kumar. (1998a). “A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs.” SIAM J. Sci. Comput. 20(1), 359–392.CrossRefGoogle Scholar
  34. Karypis, G. and V. Kumar. (1998b). “Multilevel k-way Partitioning Scheme for Irregular Graphs.” J. Parallel Distrib. Comput. 48(1), 96–129.Google Scholar
  35. Kaveh, A. and H.A. Rahimi-Bondarabady. (2000). “A Hybrid Graph-Genetic Method for Domain Decomposition.” In B.H.V. Topping (ed.), Computational Engineering Using Metaphors from Nature, Proc. of Engrg. Comput. Technology, Leuven, Belgium, Edinburgh: Civil-Comp Press, pp. 127–134.Google Scholar
  36. Kernighan, B.W. and S. Lin. (1970). “An Efficient Heuristic for Partitioning Graphs.” Bell Syst. Tech. J. 49, 291–308.Google Scholar
  37. Koren, Y. and D. Harel. (2002). “A Multi-Scale Algorithm for the Linear Arrangement Problem.” Technical Report MCS02-04, Faculty Maths. Comp. Sci., Weizmann Inst. Sci., Rehovot, Israel.Google Scholar
  38. Langham, A.E. and P.W. Grant. (1999). “A Multilevel k-way Partitioning Algorithm for Finite Element Meshes using Competing Ant Colonies.” In W. Banzhaf et al. (eds.), Proc. Genetic and Evolutionary Comput. Conf. (GECCO-1999). San Francisco: Morgan Kaufmann, pp. 1602–1608.Google Scholar
  39. Leighton, F.T. (1979). “A Graph Colouring Algorithm for Large Scheduling Problems.” J. Res. National Bureau Standards 84, 489–503.Google Scholar
  40. Lewandowski, G. and A. Condon. (1996). “Experiments with Parallel Graph Coloring and Applications of Graph Coloring.” In D.S. Johnson and M.A. Trick (eds.), Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS, pp. 309–334.Google Scholar
  41. Lin, S. (1965). “Computer Solutions of the Traveling Salesman Problem.” Bell Syst. Tech. J. 44, 2245–2269.Google Scholar
  42. Lin, S. and B.W. Kernighan. (1973). “An Effective Heuristic for the Traveling Salesman Problem.” Oper. Res. 21(2), 498–516.Google Scholar
  43. Lund, C. and M. Yannakakis. (1994). “On the Hardness of Approximating Minimization Problems.” J. ACM 41(5), 960–981.CrossRefGoogle Scholar
  44. Martin, O.C., S.W. Otto, and E.W. Felten. (1991). “Large-Step Markov Chains for the Traveling Salesman Problem.” Complex Systems 5(3), 299–326.Google Scholar
  45. Matula, D.W., G. Marble, and J.D. Isaacson. (1972). “Graph Coloring Algorithms.” In R.C. Read (ed.), Graph Theory and Computing. New York: Academic Press, pp. 109–122.Google Scholar
  46. Monien, B., R. Preis, and R. Diekmann. (2000). “Quality Matching and Local Improvement for Multilevel Graph-Partitioning.” Parallel Comput. 26(12), 1605–1634.CrossRefGoogle Scholar
  47. Neto, D.M. (1999). “Efficient Cluster Compensation for Lin-Kernighan Heuristics.” Ph.D. Thesis, Dept. Comp. Sci., University Toronto, Canada.Google Scholar
  48. Pellegrini, F. and J. Roman. (1996). “ SCOTCH: A Software Package for Static Mapping by Dual Recursive Bipartitioning of Process and Architecture Graphs.” In H. Liddell et al. (eds.), High-Performance Computing and Networking, Proc. HPCN'96, Brussels, Lecture Notes in Computer Science, Vol. 1067. Berlin: Springer, pp. 493–498.Google Scholar
  49. Reinelt, G. (1991). “TSPLIB-A Traveling Salesman Problem Library.” ORSA J. Comput. 3(4), 376–384.Google Scholar
  50. Romeijn, H.E. and R.L. Smith. (1999). “Parallel Algorithms for Solving Aggregated Shortest-Path Problems.” Comput. Oper. Res. 26(10-11), 941–953.Google Scholar
  51. Schloegel, K., G. Karypis, and V. Kumar. (2004). “Graph Partitioning for High Performance Scientific Simulations.” In J.J. Dongarra et al. (eds.), CRPC Parallel Computing Handbook, to appear. Available from partitioning.htmlGoogle Scholar
  52. Sewell, E.C. (1996). “An Improved Algorithm for Exact Graph Coloring.” In D.S. Johnson and M.A. Trick (eds.), Cliques, Coloring, and Satisfiability, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol. 26. Providence, RI: AMS, pp. 359–373.Google Scholar
  53. Simon, H.D. (1991). “Partitioning of Unstructured Problems for Parallel Processing.” Computing Systems Engrg. 2, 135–148.Google Scholar
  54. Simon, H.D. and S.-H. Teng. (1997). “How Good is Recursive Bisection?" SIAM J. Sci. Comput. 18(5), 1436–1445.CrossRefGoogle Scholar
  55. Soper, A.J., C. Walshaw, and M. Cross. (2000). “A Combined Evolutionary Search and Multilevel Optimisation Approach to Graph Partitioning.” Technical Report 00/IM/58, Comp. Math. Sci., University Greenwich, London, UK, April, to appear in J. Global Optimization.Google Scholar
  56. Teng, S.-H. (1999). “Coarsening, Sampling, and Smoothing: Elements of the Multilevel Method.” In M.T. Heath et al. (eds.), Algorithms for Parallel Processing, IMA Volumes in Mathematics and its Applications, Vol. 105. New York: Springer, pp. 247–276.Google Scholar
  57. Toulouse, M., K. Thulasiraman, and F. Glover. (1999). “Multi-level Cooperative Search: A New Paradigm for Combinatorial Optimization and an Application to Graph Partitioning.” In P. Amestoy et al. (eds.), Proc. Euro-Par'99 Parallel Processing, Lecture Notes in Computer Science, Vol. 1685. Berlin: Springer, pp. 533–542.Google Scholar
  58. Vanderstraeten, D., C. Farhat, P.S. Chen, R. Keunings, and O. Zone. (1996). “A Retrofit Based Methodology for the Fast Generation and Optimization of Large-Scale Mesh Partitions: Beyond the Minimum Interface Size Criterion.” Comput. Methods Appl. Mech. Engrg. 133, 25–45.CrossRefGoogle Scholar
  59. Walshaw, C. (2001a). “A Multilevel Algorithm for Force-Directed Graph Drawing.” In J. Marks (ed.), Graph Drawing, 8th Intl. Symp. GD 2000, Lecture Notes in Computer Science, Vol. 1984. Berlin: Springer, pp. 171–182.Google Scholar
  60. Walshaw, C. (2001b). “A Multilevel Approach to the Graph Colouring Problem.” Technical Report 01/IM/69, Comp. Math. Sci., University Greenwich, London, UK, May.Google Scholar
  61. Walshaw, C. (2001c). “A Multilevel Lin-Kernighan-Helsgaun Algorithm for the Travelling Salesman Problem.” Technical Report 01/IM/80, Comp. Math. Sci., University Greenwich, London, UK, September.Google Scholar
  62. Walshaw, C. (2001d). “Multilevel Refinement for Combinatorial Optimisation Problems.” Technical Report 01/IM/73, Comp. Math. Sci., University Greenwich, London, UK, June.Google Scholar
  63. Walshaw, C. (2002). “A Multilevel Approach to the Travelling Salesman Problem.” Oper. Res. 50(5). (Originally published as University Greenwich Technical Report 00/IM/63.)Google Scholar
  64. Walshaw, C. and M. Cross. (2000). “Mesh Partitioning: A Multilevel Balancing and Refinement Algorithm.” SIAM J. Sci. Comput. 22(1), 63–80. (Originally published as Univ. Greenwich Technical Report 98/IM/35.)CrossRefGoogle Scholar
  65. Walshaw, C., M. Cross, R. Diekmann, and F. Schlimbach. (1999). “Multilevel Mesh Partitioning for Optimising Domain Shape.” Intl. J. High Performance Comput. Appl. 13(4), 334–353. (Originally published as University Greenwich Technical Report 98/IM/38.)Google Scholar
  66. Walshaw, C. and M.G. Everett. (2002). “Multilevel Landscapes in Combinatorial Optimisation.” Technical Report 02/IM/93, Comp. Math. Sci., University Greenwich, London, UK, April.Google Scholar

Copyright information

© Kluwer Academic Publishers 2004

Authors and Affiliations

  • Chris Walshaw
    • 1
  1. 1.Computing and Mathematical SciencesUniversity of Greenwich, Old Royal Naval CollegeGreenwichUK

Personalised recommendations