Skip to main content
SpringerLink
Log in

Decomposition Algorithms for Communication Minimization in Parallel Computing

  • Chapter
Nonlinear Assignment Problems

Part of the book series: Combinatorial Optimization ((COOP,volume 7))

  • 560 Accesses

Abstract

We present algorithms that minimize the communication overheads in large classes of parallel computing applications of scientific computing and engineering. Communication minimization is essentially equivalent to graph partitioning which in turn can be formulated as quadratic assignment. Decomposition — coordination techniques using genetic algorithms as multi-coordinators have allowed us to solve problems including millions or even billions of variables in a few seconds on a network of workstations. Furthermore, we establish the asymptotically optimal behavior of these algorithms for some regular types of domains.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. (1991). The Connection Machine CM-5 Technical Summary. Thinking Machines Corporation, Cambridge, MA.

    Google Scholar 

  2. (1998). Using the CPLEX 6.0 Callable Library. ILOG CPLEX Division, 889 Alder Ave. Incline Village, NV 89451.

    Google Scholar 

  3. Ahuja, R. K., Magnanti, T. L., and Orlin, J. B. (1993). Network Flows. Prentice Hall, Englewood Cliffs, NJ.

    MATH  Google Scholar 

  4. Chen, R., Meyer, R., and Yackel, J. (1993). A genetic algorithm for diversity minimization and its parallel implementation. In Proceedings of the Fifth International Conference on Genetic Algorithms, pages 163–170, Palo Alto, CA. Morgan Kaufman.

    Google Scholar 

  5. Christou, I., Martin, W., and Meyer, R. R. (1999a). Genetic algorithms as multi-coordinators in large scale optimization. In Davis, L. D., Whitley, L. D., DeJong, K., and Vose, M., editors, Evolutionary Algorithms, volume 111, pages 1–15, New York, NY. Springer-Verlag. The IMA Volumes in Mathematics and its Applications.

    Chapter  Google Scholar 

  6. Christou, I. T. (1996). Distributed Genetic Algorithms for Partitioning Uniform Grids. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison, Madison, WI.

    Google Scholar 

  7. Christou, I. T. and Meyer, R. R. (1996a). Fast distributed genetic algorithms for partitioning uniform grids. In Rolim, H. and Yang, T., editors, Lecture Notes in Computer Science, volume 1117, New York, NY. Springer-Verlag. Proceedings of the Third International Workshop on Parallel Algorithms for Irregularly Structured Problems (IRREGULAR 96).

    Google Scholar 

  8. Christou, I. T. and Meyer, R. R. (1996b). Optimal and asymptotically optimal equi-partition of rectangular domains via stripe decomposition. In Fischer, H., Riedmuller, B., and Schaffler, S., editors, Applied Mathematics and Parallel Computing — Festschrift for Klaus Ritter, pages 77–96, Berlin, Germany. Physica-Verlag.

    Chapter  Google Scholar 

  9. Christou, I. T. and Meyer, R. R. (1996c). Optimal equi-partition of rectangular domains for parallel computation. Journal of Global Optimization, 8:15–34.

    Article  MathSciNet  MATH  Google Scholar 

  10. Christou, I. T., Zakarian, A., Liu, J., and Carter, H. (1999b). A two-phase genetic algorithm for large scale bidline generation problems at delta air lines. Interfaces, 29(5).

    Google Scholar 

  11. Crandall, P. and Quinn, M. (1995). Non-uniform 2-d grid partitioning for heterogeneous parallel architectures. In Proceedings of the 9th International Symposium on Parallel Processing, pages 428–435, Los Alamitos, CA. IEEE Computer Society Press.

    Chapter  Google Scholar 

  12. Dantzig, G. B. and Wolfe, P. (1960). Decomposition principle for linear programs. Operations Research, 8:101–111.

    Article  MATH  Google Scholar 

  13. Darwin, C. (1859). On the Origin of Species by Means of Natural Selection. J. Murray, London, UK.

    Google Scholar 

  14. DeLeone, R., Meyer, R. R., Kontogiorgis, S., Zakarian, A., and Zakeri, G. (1994). Coordination in coarse-grain decomposition. SIAM Journal on Optimization, 4(4):773–793.

    MathSciNet  Google Scholar 

  15. Falkenauer, E. (1998). Genetic Algorithms and Grouping Problems. John Wiley & Sons, New York, NY.

    Google Scholar 

  16. Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslovak Math. Journal, 23:298–305.

    MathSciNet  Google Scholar 

  17. Fulkerson, D. R. (1963). Flows in networks. In Recent Advances in Mathematical Programming, pages 319–332, New York, NY. McGrow — Hill.

    Google Scholar 

  18. Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, R., and Sunderam, V. (1994). PVM 3 User’s Guide and Reference Manual. Oak Ridge National Laboratory, Oak Ridge, TN.

    Google Scholar 

  19. Gilbert, J. R., Miller, G. L., and Teng, S. H. (1995). Geometric mesh partitioning: Implementation and experiments. In Proceedings of the 9th International Symposium on Parallel Processing, pages 418–427, Los Alamitos, CA. IEEE Computer Society Press.

    Chapter  Google Scholar 

  20. Hart, W. E. (1994). Adaptive global optimization with local search. PhD thesis, Computer Science and Engineering Dept. University of California — San Diego, San Diego, CA.

    Google Scholar 

  21. Hendrickson, B. and Leland, R. (1993). Multidimensional spectral load balancing. In Proceedings of the 6th SIAM Conference on Parallel Processing and Scientific Computing, Philadelphia, PA. SIAM.

    Google Scholar 

  22. Hendrickson, B. and Leland, R. (1995a). The Chaco User’s Guide Version 2.0. Sandia National Laboratories, Sandia, NM.

    Google Scholar 

  23. Hendrickson, B. and Leland, R. (1995b). An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM Journal on Scientific Computation, 16:452–469.

    Article  MathSciNet  MATH  Google Scholar 

  24. Holland, J. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Cambridge, MA.

    Google Scholar 

  25. Karypis, G. and Kumar, V. (1995). Multilevel kway partitioning scheme for irregular graphs. Technical Report 95–064, Computer Science Dept. University of Minnesota.

    Google Scholar 

  26. Kernighan, B. W. and Lin, S. (1970). An effective heuristic procedure for partitioning graphs. Bell Systems Tech. Journal, pages 291–308.

    Google Scholar 

  27. Laguna, M., Feo, T. A., and Elrod, H. C. (1994). A greedy randomized adaptive search procedure for the two — partition problem. Operations Research.

    Google Scholar 

  28. Levine, D. (1995). User’s Guide to the PGAPack Parallel Genetic Algorithm Library Version 0.2. Argonne National Laboratory, Argonne, IL.

    Google Scholar 

  29. Li, Y., Pardalos, P. M., and Resende, M. G. C. (1994). A greedy randomized adaptive search procedure for the quadratic assignment problem. In Pardalos, P. M. and Wolkowicz, H., editors, Quadratic Assignment and Related Problems, pages 237–262, Providence, RI. American Mathematical Society.

    Google Scholar 

  30. Lin, K. Y. (1991). Exact solution of the convex polygon perimeter and area generating function. J. Phys. A. Math Gen., 24:2411–2417.

    Article  Google Scholar 

  31. Martin, W. (1998). Fast equi-partitioning of rectangular domains using stripe decomposition. Discrete Applied Mathematics, 82(1– 3): 193–207.

    Article  MathSciNet  MATH  Google Scholar 

  32. Melou, M. B. (1994). Codage des polyominos convexes et equation pour l’enumeration suivant l’aire. Discrete Applied Mathematics, 48:21–43.

    Article  MathSciNet  MATH  Google Scholar 

  33. Michalewicz, Z. (1994). Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York, NY.

    MATH  Google Scholar 

  34. Miller, G. L., Teng, S. H., Thurston, W., and Vavasis, S. A. (1993). Automatic mesh partitioning. In George, A., Gilbert, J. R., and Liu, J. W. H., editors, Graph Theory and Sparse Matrix Computation, New York, NY. Springer-Verlag.

    Google Scholar 

  35. Muhlenbein, H., Schomisch, M., and Born, J. (1991). The parallel genetic algorithm as function optimizer. In Belew, R. and Booker, L., editors, Proceedings of the Fourth Intl. Conference on Genetic Algorithms, pages 45–52, Los Altos, CA. Morgan Kaufmann Publishers.

    Google Scholar 

  36. Nemhauser, G. and Wolsey, L. (1985). Integer and Combinatorial Optimization. John Wiley & Sons, New York, NY.

    Google Scholar 

  37. Pardalos, P., Pitsoulis, L., and Resende, M. (1995). A parallel grasp implementation for the quadratic assignment problem. In Ferreira, A. and Rolim, J., editors, Parallel Algorithms for Irregularly Structured Problems, pages 111–130, Dordrecht, The Netherlands. Kluwer Academic Publishers. Proceedings of the First International Workshop on Parallel A1gorithms for Irregularly Structured Problems (IRREGULAR 94).

    Google Scholar 

  38. Pardalos, P. M., Rendl, F., and Wolkowicz, H. (1993). The quadratic assignment problem: A survey and recent developments. In Pardalos, P. M. and Wolkowicz, H., editors, QuadraticAssignmentandRelated Problems, pages 1–42, Providence, RI. American Mathematical Society.

    Google Scholar 

  39. Pothen, A., Simon, H. D., and Liu, K. P. (1990). Partitioning sparse matrices with eigenvectors of graphs. SIAM Journal on Matrix Analysis and Applications, 11:430–452.

    Article  MathSciNet  MATH  Google Scholar 

  40. Schalkoff, R. J. (1989). Digital Image Processing and Computer Vision. John Wiley & Sons, New York, NY.

    Google Scholar 

  41. Schultz, G. L. and Meyer, R. R. (1991). An interior point method for block angular optimization. SIAM Journal on Optimization, 1(4) : 5 8 3–602.

    Google Scholar 

  42. Strikwerda, J. (1989). Finite Difference Schemes and Partial Differential Equations. Wadsworth & Brooks Cole, Pacific Grove, CA.

    MATH  Google Scholar 

  43. von Laszewski, G. (1991). Intelligent structural operators for the k-way graph partitioning problem. In Belew, R. and Booker, L., editors, Proceedings of the Fourth Intl. Conference on Genetic Algorithms, pages 45–52, Los Altos, CA. Morgan Kaufmann Publishers.

    Google Scholar 

  44. Yackel, J. (1993). Minimum Perimeter Tiling in Parallel Computation. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison, Madison, WI.

    Google Scholar 

  45. Yackel, J. and Meyer, R. R. (1992a). Minimum perimeter decomposition. Technical Report 1078, University of Wisconsin — Madison, Madison, WI.

    Google Scholar 

  46. Yackel, J. and Meyer, R. R. (1992b). Optimal tilings for parallel database design. In Pardalos, P. M., editor, Advances in Optimization and Parallel Computing, pages 293–309, New York, NY. North — Holland.

    Google Scholar 

  47. Yackel, J., Meyer, R. R., and Christou, I. T. (1997). Minimum-perimeter domain assignment. Mathematical Programming, 78:283–303.

    MathSciNet  MATH  Google Scholar 

  48. Zakarian, A. (1995). NonLinear Jacobi and epsilon — Relaxation Methods for Parallel Network Optimization. PhD thesis, Computer Sciences Dept. University of Wisconsin — Madison.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. of Operations Research, Delta Technology Inc., 1001 International Blvd., Atlanta, GA, 30354, USA

    Ioannis T. Christou

  2. Computer Sciences Dept., University of Wisconsin - Madison, 1201 W. Dayton Str., Madison, 53706, WI, USA

    Robert R. Meyer

Authors
  1. Ioannis T. Christou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert R. Meyer
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Florida, USA

    Panos M. Pardalos

  2. Princeton University, USA

    Leonidas S. Pitsoulis

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Christou, I.T., Meyer, R.R. (2000). Decomposition Algorithms for Communication Minimization in Parallel Computing. In: Pardalos, P.M., Pitsoulis, L.S. (eds) Nonlinear Assignment Problems. Combinatorial Optimization, vol 7. Springer, Boston, MA. https://doi.org/10.1007/978-1-4757-3155-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-1-4757-3155-2_10

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4419-4841-0

  • Online ISBN: 978-1-4757-3155-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation