Hybrid Parallel Programming on SMP Clusters Using XPFortran and OpenMP
Process-thread hybrid programming paradigm is commonly employed in SMP clusters. XPFortran, a parallel programming language that specifies a set of compiler directives and library routines, can be used to realize process-level parallelism in distributed memory systems. In this paper, we introduce hybrid parallel programming by XPFortran to SMP clusters, in which thread-level parallelism is realized by OpenMP. We present the language support and compiler implementation of OpenMP directives in XPFortran, and show some of our experiences in XPFortran-OpenMP hybrid programming. For nested loops parallelized by process-thread hybrid programming, it’s common sense to use process parallelization for outer loops and thread parallelization for inner ones. However, we have found that in some cases it’s possible to write XPFortran-OpenMP hybrid program in a reverse way, i.e., OpenMP outside, XPFortran inside. Our evaluation results show that this programming style sometimes delivers better performance than the traditional one. We therefore recommend using the hybrid parallelization flexibly.
KeywordsNest Loop Hybrid Programming Fortran Compiler Process Parallelization High Performance Fortran
Unable to display preview. Download preview PDF.
- 1.Co-Array Fortran, http://www.co-array.org/
- 2.Himeno Benchmark Program, http://accc.riken.jp/HPC_e/HimenoBMT_e.html
- 3.EPCC OpenMP Microbenchmarks, http://www2.epcc.ed.ac.uk/computing/research_activities/openmpbench/openmp_index.html
- 4.Message Passing Interface Forum. MPI: A message-passing interface standard, http://www.mpi-forum.org/
- 5.OpenMP, http://openmp.org/wp/
- 7.Iwashita, H., Sueyasu, N., Kamiya, S., van Waveren, M.: VPP Fortran and the Design of HPF/JA Extensions. In: Concurrency and Computation: Practice and Experience, vol. 14(8-9), pp. 575–588. John Wiley &Sons Ltd., Chichester (2002)Google Scholar
- 8.Iwashita, H., Aoki, M.: A Code Generation Technique Common to Distribution Kinds on the HPF Translator. IPSJ Transactions on Advanced Computing Systems (ACS) 0(15), 329–339 (2006) (in Japanese) Google Scholar
- 9.Rabenseifner, R., Hager, G., Jost, G.: Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes. In: Proc. of the 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing, pp. 427–436 (2009)Google Scholar