An Automatic Iteration/Data Distribution Method Based on Access Descriptors for DSMM

Abstract

Nowadays NUMA architectures are widely accepted. For such multiprocessors exploiting data locality is clearly a key issue. In this work, we present a method for automatically selecting the iteration/data distributions for a sequential F77 code, while minimizing the parallel execution overhead (communications and load unbalance). We formulate an integer programming problem to achieve that minimum parallel overhead. The constraints of the integer programming problem are derived directly from a graph known as the Locality-Communication Graph (LCG), which captures the memory locality, as well as the communication patterns, of a parallel program. In addition, our approach use the LCG to automatically schedule the communication operations required during the program execution, once the iteration/data distributions have been selected. The aggregation of messages in blocks is also dealt in our approach. The TFFT2 code, from NASA benchmarks, that includes non-affine access functions and non-affine index bounds, and repeated subroutine calls inside loops, has been correctly handled by our approach. With the iteration/data distributions derived from our method, this code achieves parallel efficiencies of over 69% for 16 processors, in a Cray T3E, an excellent performance for a complex real code.

This work was supported by the Ministry of Education and Science (CICYT) of Spain (TIC96-1125-C03) and by the European Union (EU ESPRIT IV Working Group No. 29488)