Skip to main content

Parallel physical optimization algorithms for allocating data to multicomputer nodes


Three parallel physical optimization algorithms for allocating irregular data to multicomputer nodes are presented. They are based on simulated annealing, neural networks and genetic algorithms. All three algorithms deviate from the sequential versions in order to achieve acceptable speedups. The parallel simulated annealing (PSA) and neural network (PNN) algorithms include communication schemes that are adapted to the properties of the allocation problem and of the algorithms themselves for maintaining both good solutions and reasonable execution times. The parallel genetic algorithm (PGA) is based on a natural model of evolution. The performances of these algorithms are evaluated and compared. The three parallel algorithms maintain the good solution qualities of their sequential counterparts. Their comparison shows their suitability for different applications. For example, PGA yields the best solutions, but it is the slowest of the three. PNN is the fastest, but it yields lower quality solutions. PSA's performance lies in the middle.

This is a preview of subscription content, access via your institution.


  1. Aarts, E., and Korst, J. 1989.Simulated Annealing and Boltzmann Machines. New York, Wiley.

    Google Scholar 

  2. Berger, M., and Bokhari, S. 1987. A partitioning strategy for nonuniform problems on multiprocessors.IEEE Trans. Comps., C-36, 5(May):570–580.

    Google Scholar 

  3. Bout, D.E. Van den, and Miller, T.K. 1990. Graph partitioning using annealed neural networks.IEEE Trans. Neural Networks, 1,2:192–203.

    Google Scholar 

  4. Byun, H., Kortesis, S.K., and Houstis, E.N. 1992. A workload partitioning strategy for PDEs by a generalized neural network. Purdue Univ., Tech. Rept. CSD-TR-92-015.

  5. Chrisochoides, N.P., Houstis, E.N., and Houstis, C.E. 1991. Geometry based mapping strategies for PDE computations. InInternat. Conf. on Supercomputing, ACM Press, pp. 115–127.

  6. Eglese, R.W. 1990. Simulated annealing: A tool for operational research.Eur. J. Operational Research, 46:271–281.

    Google Scholar 

  7. Ercal, F. 1988. Heuristic approaches to task allocation for parallel computing. Ph.D. thesis, Ohio State Univ., Columbus, Oh.

    Google Scholar 

  8. Flower, J., Otto, S., and Salama, M. 1987. A preprocessor for finite element problems. InProc., Symp. on Parallel Computations and Their Impact on Mechanics, ASME Winter Meeting (Dec).

  9. Fox, G.C. 1988. A graphical approach to load balancing and sparse matrix vector multiplication on the hypercube. InNumerical Algorithms for Modern Parallel Computers (M. Schultz, ed.), Berlin, Springer-Verlag.

    Google Scholar 

  10. Fox, G.C. 1990. Physical computation. InProc., Internat. Conf. on Parallel Computing: Achievements, Problems and Prospects (Italy, June).

  11. Fox, G.C., and Furmanski, W. 1988. Load balancing loosely synchronous problems with a neural network. InProc., 3rd Conf. on Hypercube Concurrent Computers, and Applications, pp. 241–278.

  12. Fox, G.C., Johnson, M., Lyzenga, G., Otto, S., Salmon, J., and Walker, D. 1988.Solving Problems on Concurrent Processors. Englewood Cliffs, N.J., Prentice-Hall.

    Google Scholar 

  13. Fox, G.C., Hiranandani, S., Kennedy, K., Koelbel, C., Kremer, U., Tseng, C., and Wu, M.-Y. 1990. Fortran D language specification. Syracuse Univ., NPAC, SCCS-42.

  14. Garey, M.R., and Johnson, D.S. 1979.Computers and Intractability. Freeman.

  15. Goldberg, D.E. 1989.Genetic Algorithms in Search, Optimization and Machine Learning. Reading, Mass., Addison-Wesley.

    Google Scholar 

  16. Greening, D.R. 1990. Parallel simulated annealing techniques.Physica D, 42:293–306.

    Google Scholar 

  17. Holland, J.H. 1975.Adaptation in Natural and Artificial Systems. Univ. of Michigan Press, Ann Arbor.

    Google Scholar 

  18. Hopfield, J.J., and Tank, D.W. 1986. Computing with neural circuits: A model.Science, 233:625–639.

    Google Scholar 

  19. Houstis, E.N., Rice, J.R., Chrisochoides, N.P., Karathonases, H.C., Papachiou, P.N., Samartzis, M.K., Vavalis, E.A., Wang, K. Y., and Weerawarana, S. 1990.//ELLPACK: A numerical simulation programming environment for parallel MIMD machines. InInternat. Conf. on Supercomputing, ACM Press, pp. 3–23.

  20. Johnson, D.S., Aragon, C.R., McGeoch, L.A., and Schevon, C. 1989. Optimization by simulated annealing: An experimental evaluation; part 1, graph partitioning.Operations Research, 37,6:865–892.

    Google Scholar 

  21. Kirkpatrick, S., Gelatt, C.D., and Vecchi, M.P. 1983. Optimization by simulated annealing.Science, 220:671–680.

    Google Scholar 

  22. Mansour, N. 1992. Physical optimization algorithms for mapping data to distributed-memory multiprocessors. Ph.D. diss., Comp. Sci., Syracuse Univ., Syracuse, N.Y.

    Google Scholar 

  23. Mansour, N., and Fox, G.C. 1991. A hybrid genetic algorithm for task allocation. InProc., Internat. Conf. on Genetic Algorithms (July), pp. 466–473.

  24. Mansour, N., and Fox, G.C. 1992a. Allocating data to multicomputer nodes by physical optimization algorithms for loosely synchronous computations.Concurrency: Practice and Experience, 4,7:557–574.

    Google Scholar 

  25. Mansour, N., and Fox, G.C. 1992b. Parallel genetic algorithms with application to load balancing parallel computations. InSupercomputing Symp. (Montreal, June), Atmospheric Environment Services, pp. 128–139.

    Google Scholar 

  26. Nolting, S. 1991. Nonlinear adaptive finite element systems on distributed memory computers. InEur. Distributed Memory Computing Conf. (Apr.), pp. 283–293.

  27. Pothen, A., Simon, H., and Liou, K.-P. 1990. Partitioning sparse matrices with eigenvectors of graphs.SIAM J. Matrix Anal. Appl., 11, 3 (July):430–452.

    Google Scholar 

  28. Raman, S., and Patnaik, L.M. 1990. Annealing-based circuit partitioner for hypercube architectures.Internat. J. High Speed Computing, 2,1:69–84.

    Google Scholar 

  29. Simon, H. 1991. Partitioning of unstructured mesh problems for parallel processing. InProc., Conf. on Parallel Methods on Large Scale Structural Analysis and Physics Applications, Permagon Press.

  30. Walker, D. 1990. Characterizing the parallel performance of a large-scale, particle-in-cell plasma simulation code.Concurrency: Practice and Experience (Dec.):257–288.

  31. Williams, R.D. 1991. Performance of dynamic load balancing algorithms for unstructured mesh calculations.Concurrency: Practice and Experience, 3,5:457–481.

    Google Scholar 

  32. Wright, S. 1977.Evolution and the Genetics of Populations, vol. 3. Univ. of Chicago Press.

Download references

Author information



Rights and permissions

Reprints and Permissions

About this article

Cite this article

Mansour, N., Fox, G.C. Parallel physical optimization algorithms for allocating data to multicomputer nodes. J Supercomput 8, 53–80 (1994).

Download citation


  • Automatic parallelization
  • data allocation
  • data partitioning
  • genetic algorithms
  • load balancing
  • mapping
  • neural networks
  • physical optimization
  • simulated annealing