International Journal of Parallel Programming

, Volume 35, Issue 1, pp 33–61 | Cite as

Memetic Algorithms for Parallel Code Optimization

  • Ender Özcan
  • Esin OnbaşioğluEmail author

Discovering the optimum number of processors and the distribution of data on distributed memory parallel computers for a given algorithm is a demanding task. A memetic algorithm (MA) is proposed here to find the best number of processors and the best data distribution method to be used for each stage of a parallel program. Steady state memetic algorithm is compared with transgenerational memetic algorithm using different crossover operators and hill-climbing methods. A self-adaptive MA is also implemented, based on a multimeme strategy. All the experiments are carried out on computationally intensive, communication intensive, and mixed problem instances. The MA performs successfully for the illustrative problem instances.


Distributed memory parallel computers memetic algorithms parallelizing compilers search methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson J., Lam M., (1993) Global Optimizations for Parallelism and Locality on Scalable Parallel Machines, in Proc. of the ACM SIGPLAN Conf. on Programming Language Design and Implementation, 112–125 (1993).Google Scholar
  2. 2.
    P. Banerjee, J. Chandy, M. Gupta, E. Hodge, J. Holm, A. Lain, D. Palermo, S. Ramaswamy, and E. Su, The Paradigm Compiler for Distributed-Memory Multicomputers, IEEE Comput. October: 37–47 (1995).Google Scholar
  3. 3.
    Beaumont O., Boudet V., Petitet A., Rastello F., Robert Y. (2001) A Proposal for Heterogeneous Cluster Scalapack. IEEE Trans. Comput. 50(10):1052–1071CrossRefMathSciNetGoogle Scholar
  4. 4.
    Beaumont O., Boudet V., Rastello F., Robert Y., (2001) Matrix Multiplication on Heterogeneous Platforms, IEEE Trans. Parall. Distri. Syst. 12(10):1033–1051CrossRefMathSciNetGoogle Scholar
  5. 5.
    R. Bixby, K. Kennedy, and U. Kremer, Automatic Data Layout using 0-1 Integer Programming, in Proc. Int. Conf. on Parallel Architectures and Compilation Techniques (PACT’94) pp. 111–122 (1994).Google Scholar
  6. 6.
    H. Bremermann, Optimization Through Evolution and Recombination, Self-Organizing Syst. 93–106 (1962).Google Scholar
  7. 7.
    Chen T., Sheu J. (1994) Communication-Free Data Allocation Techniques for Parallelizing Compilers on Multicomputers, IEEE Trans. Parall. Distri. Syst. 5:924–938CrossRefGoogle Scholar
  8. 8.
    J. Choi, J. Dongarra, and D. W. Walker, The design of a parallel, Dense Linear Algebra Software Library: Reduction to Hessenberg, Tridiagonal and Bidiagonal Form, in Proc. of the 2nd Workshop on Environments and Tools for Parallel Scientific Computing, pp. 98–111 (1994).Google Scholar
  9. 9.
    H.G. Cobb, An investigation into the use of hypermutation as an adaptive operator in Genetic Algorithms Having Continuous, Time-dependent Nonstationary Environment, NRL Memorandum Report 6760 (1990).Google Scholar
  10. 10.
    K.A. De Jong, An Analysis of the Behavior of a Class of Genetic Adaptive Systems, PhD Thesis, University of Michigan (1975).Google Scholar
  11. 11.
    Fraser A. (1957) Simulation of Genetic Systems by Automatic Digital Computers, 2 Vols, Aust. J Biol. Sci. 10:484–491Google Scholar
  12. 12.
    Friedman G.J. (1959) Digital Simulation of An Evolutionary Process. Gen. Syst. Yearbook 4:171–184Google Scholar
  13. 13.
    Goldberg D.E. (1989) Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, MAzbMATHGoogle Scholar
  14. 14.
    Holland J. H. Adaptation in Natural and Artificial Systems. University of Michican Press, (1975).Google Scholar
  15. 15.
    Huang C.H., Sadayappan P. (1993) Communication-Free Hyperplane Partitioning of Nested Loops. J. Parall. and Distr. Comput. 19:90–102zbMATHCrossRefGoogle Scholar
  16. 16.
    K. Ikudome, G. Fox, A. Kolawa, and J. Flower, An Automatic and Symbolic Parallelization System for Distributed Memory Parallel Computers, in Proc. of 5th Distributed Memory Computing Conference, pp.1105–1114 (1990).Google Scholar
  17. 17.
    Karp A.H. (1987) Programming for Parallelism. IEEE Comput. 20(5):43–57MathSciNetGoogle Scholar
  18. 18.
    Knobe K., Lucas J., Steele G. (1990). Data Optimizations: Allocation of Arrays to Reduce Communication on SIMD machines. J. arall. Distr. Compu. 8(2):102–118CrossRefGoogle Scholar
  19. 19.
    N. Krasnogor, Studies on the Theory and Design Space of Memetic Algorithms, PhD Thesis, University of the West of England (2002).Google Scholar
  20. 20.
    N. Krasnogor and J. Smith, Emergence of Profitable Search Strategies Based on a Simple Inheritance Mechanism, in Proc. of the 2001 International Genetic and Evolutionary Computation Conference (GECCO2001) (2001).Google Scholar
  21. 21.
    Krasnogor N., Gustafson S. (2004) A Study on the use of “Self-Generation” in Memetic Algorithms. Nat. Comput. 3(1):53–76zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Lee P. (1997) Efficient algorithms for Data Distribution on Distributed Memory Parallel Computers. IEEE Trans. Parall. Distr. Syst. 8(8):825–839CrossRefGoogle Scholar
  23. 23.
    J. Li, and M. Chen, Index Domain Alignment: Minimizing Costs of Cross-referencing between Distributed Arrays, in Proc. of the Third Symp. on the Frontiers of Massively Parallel Computation, pp. 424–433 (1990).Google Scholar
  24. 24.
    Mace M. (1987) Memory Storage Patterns in Parallel Processing. Kluwer Academic, DordrechtGoogle Scholar
  25. 25.
    Mitchell M., Forrest S. (1994) Genetic Algorithms and Artificial Life. Artificial Life 1(3):267–289CrossRefGoogle Scholar
  26. 26.
    J. Mohan, Performance of Parallel Programs, PhD dissertation, Dept. of Computer Science, Carnegie-Mellon University, (1984).Google Scholar
  27. 27.
    P. Moscato and M. G. Norman, A Memetic Approach for the Traveling Salesman Problem Implementation of a Computational Ecology for Combinatorial Optimization on Message-Passing Systems, Parall. Comput. Transputer Appl. 177–186 (1992).Google Scholar
  28. 28.
    Onbasioglu E., Ozdamar L. (2003) Optimization of Data Distribution and Processor Allocation Problem using Simulated Annealing. J. Supercomp. 25:237–253zbMATHCrossRefGoogle Scholar
  29. 29.
    Onbasioglu E., Paker Y. (1997). A Comparative Workload-based Methodology for Performance Evaluation of Parallel Computers. Future Gen. Comput. Syst. 12:512–545CrossRefGoogle Scholar
  30. 30.
    E. Ozcan An Empirical Investigation on Memes, Self-generation and Nurse Rostering, in Proc. of the 6th International Conference on the Practice and Theory of Automated Timetabling, 246–263 (2006).Google Scholar
  31. 31.
    E. Ozcan, Memetic Algorithms for Nurse Rostering, in P. Yolum, T. Güngör, F. Gürgen, and C. Özturan, (Eds.) Proc. of the 20th International Symposium on Computer and Information Sciences, Lecture Notes in Computer Science, Springer-Verlag. pp.482–492 (2005).Google Scholar
  32. 32.
    E. Ozcan and C. K. Mohan, Steady State Memetic Algorithm for Partial Shape Matching, V. William Porto, N. Saravanan, Donald E. Waagen, and A. E. Eiben (Eds.) in Evolutionary Programming VII. 7th International Conference, EP98, Lecture Notes in Computer Science. Springer. vol. 1447:527–536. 1998.Google Scholar
  33. 33.
    E. Ozcan and E. Onbasioglu, Genetic Algorithms for Parallel Code Optimization. in Proc. of 2004 IEEE Congress on Evolutionary Computation, 2:pp.1775–1781 (2004).Google Scholar
  34. 34.
    D. Palermo, Compiler Techniques for Optimizing Communication and Data Distribution for Distributed Memory Multicomputers, PhD thesis, University of Illinois at Urbana-Champaign (1996).Google Scholar
  35. 35.
    Prügel-Bennett A., Shapiro J.L. (1997) An Analysis of Genetic Algorithms Using Statistical Mechanics. Physica D 104:75–114CrossRefMathSciNetGoogle Scholar
  36. 36.
    N. J. Radcliffe and P. D. Surry, Formal Memetic Algorithms, Evol. Comput. AISB Workshop. 1–16 (1994).Google Scholar
  37. 37.
    Ramanujan J., Sadayappan P. (1991) Compile-time Techniques for Data Distribution in Distributed Memory Machines. IEEE Trans. Parall. Distr. Syst. 2(4):472–482CrossRefGoogle Scholar
  38. 38.
    Rauber T., Runger G. (2000) Deriving Array Distributions by Optimization Techniques. J. Supercomput. 15:271–293zbMATHCrossRefGoogle Scholar
  39. 39.
    R. Sierra, Margarita, M. and C. A. Coello Coello, On the Study of Some Theoretical Aspects of Genetic Algorithms, Technical Report EVOCINV-02–2003, Evolutionary Computation Group at CINVESTAV, Sección de Computación, Departamento de Ingeniería Eléctrica, CINVESTAV-IPN, México (2003).Google Scholar
  40. 40.
    Smith J.E., Fogarty T.C. (1997) Operator and Parameter Adaptation in Genetic Algorithms. Soft Comput. 12:81–87CrossRefGoogle Scholar
  41. 41.
    L. D. Whitley and M. D. Vose, Foundations of Genetic Algorithms 3, Morgan Kaufmann Los Altos, (1995).Google Scholar
  42. 42.
    S. Wholey, Automatic Data Mapping for Distributed Memory Parallel Computers, Proc. Int. Conf. Supercomput. pp. 25–34 (1992).Google Scholar
  43. 43.
    M.E. Wolf and M. S. Lam, A Data Locality Optimizing Algorithm. in. Proc. ACM SIGPLAN Conf. on Programming Language Design and Implementation, pp. 30–44 (1991).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2006

Authors and Affiliations

  1. 1.Bilgisayar Muhendisligi Bolumu, Inonu Mah. KAYISDAGI Cad.Yeditepe UniversitesiKadikoy/IstanbulTurkey

Personalised recommendations