Data Flow Analysis Driven Dynamic Data Partitioning

  • Jodi Tims
  • Rajiv Gupta
  • Mary Lou Soffa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1511)


The use of distributed memory architectures as an effective approach to parallel computing brings with it a more complex program development process. Finding a partitioning of program code and data that supports sufficient parallelism without incurring prohibitive communication costs is a challenging and critical step in the development of programs for distributed memory systems. Automatic data distribution techniques have the goal of placing the responsibility of determining a suitable data partitioning into the domain of the compiler. Static program analysis techniques that expose data interrelationships and derive performance estimates are central to the development of automatic data distribution heuristics. In this paper we present a data partitioning heuristic that makes use of array data flow analysis information in the modeling of data interrelationships and the estimation of costs associated with resolving interrelationships via communication. The global view provided by data flow analysis permits consideration of potential communication optimizations before data partitioning decisions are made. Our heuristic uses tiling techniques to determine data partitionings. The resulting data distributions, while still regular, are not limited to the standard BLOCK, CYCLIC and BLOCK-CYCLIC varieties. Preliminary results indicate an overall reduction in communication cost with our technique.


Communication Cost Program Point Control Flow Graph Communication Dependence Extreme Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S.P. Amarashinge, J.M. Anderson, M.S. Lam, and C.W. Tseng. “The SUIF Compiler for Scalable Parallel Machines„. In Proceedings of the Seventh SIAM Conference on Parallel Processing for Scientific Computing, Feb. 1995.Google Scholar
  2. 2.
    Jennifer M. Anderson and Monica S. Lam. “Global Optimizations for Parallelism and Locality on Scalable Parallel Machines„. In ACM SIGPLAN’ 93 Conference on Programming Language Design and Implementation, pages 112–125, Albuquerque, NM, Jun. 1993.Google Scholar
  3. 3.
    Eduard Ayguade, Jordi Garcia, Merce Girones, M. Luz Grande, and Jesus Labarta. “Data Redistribution in an Automatic Data Distribution Tool. In Proceedings of the 8th Workshop on Languages and Compilers for Parallel Computing, pages 407–421, Columbus, Ohio, Aug. 1995.Google Scholar
  4. 4.
    R. Bixby, K. Kennedy, and U. Kremer. “Automatic Data Layout Using 0-1 Integer Programming„. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT94), pages 111–122, Montreal, Canada, Aug. 1994.Google Scholar
  5. 5.
    D. Callahan and K. Kennedy. “Compiling Programs for Distributed-Memory Multiprocessors„. The Journal of SuperComputing, 2, 1988.Google Scholar
  6. 6.
    Barbara M. Chapman, Thomas Fahringer, and Hans P. Zima. “Automatic Support for Data Distribution on Distributed Memory Multiprocessor Systems„. In Proceedings of the 6th Workshop on Languages and Compilers for Parallel Computing, pages 184–199, Portland, Oregon, Aug. 1993.Google Scholar
  7. 7.
    S. Chatterjee, J. R. Gilbert, R. Schreiber, and T.J. Sheffler. “Array Distribution in Data-Parallel Programs„. In Proceedings of the 7th Workshop on Languages and Compilers for Parallel Computing, pages 76–91, 1994.Google Scholar
  8. 8.
    Ron Cytron, Jeanne Ferrante, Barry Rosen, Mark N. Wegman, and F. Kenneth Zadeck. “Efficiently Computing Static Single Assignment Form and the Control Dependence Graph„. In ACM Transactions on Programming Languages and Systems, volume 13, pages 451–490, Oct. 1991.Google Scholar
  9. 9.
    Thomas Gross and Peter Steenkiste. “Structured Dataflow Analysis for Arrays and its Use in an Optimizing Compilers„. Software Practice and Experience, 20(2):133–155, Feb. 1990.Google Scholar
  10. 10.
    Manish Gupta and Prithviraj Banerjee. “Demonstration of Automatic Data Partitioning Techniques for Parallelizing Compilers on Multicomputers„. IEEE Transactions on Parallel and Distributed Systems, 3(2):179–193, Mar. 1992.Google Scholar
  11. 11.
    Richard Johnson and Keshav Pingali. “Dependence-Based Program Analysis„. In ACM SIGPLAN’ 93 Conference on Programming Language Design and Implementation, pages 78–89, Jun. 1993.Google Scholar
  12. 12.
    M. Kandemir, J. Ramanujam, and A. Choudhary. “Compiler Algorithms for Optimizing Locality and Parallelism on Shared and Distributed Memory Machines„. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT97), San Francisco, CA, Nov. 1997.Google Scholar
  13. 13.
    Kathleen Knobe, Joan D. Lukas, and Guy L. Steele Jr. “Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines„. Journal of Parallel and Distributed Computing, 8:102–118, 1990.CrossRefGoogle Scholar
  14. 14.
    Uli Kremer. “NP-completeness of Dynamic Remapping„. In Proceedings of the Fourth Workshop on Compilers for Parallel Computers, The Netherlands, Dec. 1993.Google Scholar
  15. 15.
    Daniel J. Palermo and Prithviraj Banerjee. “Automatic Selection of Dynamic Data Partitioning Schemes for Distributed-Memory Multipcomputers„. In Proceedings of the 8th Workshop on Languages and Compilers for Parallel Computing, pages 392–406, Columbus, OH, Aug. 1996.Google Scholar
  16. 16.
    J. Ramanujam and P. Sadayappan. “Nested Loop Tiling for Distributed Memory Machines„. In 5th Distributed Memory Computing Conference, pages 1088–1096, Charleston, SC, Apr. 1990.Google Scholar
  17. 17.
    Thomas J. Sheffler, Robert Schreiber, William Pugh, John R. Gilbert, and Siddhartha Chatterjee. “Efficient Distribution Analysis via Graph Contraction„. In Frontiers’ 95: The 5th Symposium on the Frontiers of Massively Parallel Computation, pages 377–391, McLean, VA, Feb. 1995.Google Scholar
  18. 18.
    X. Yuan, R. Gupta, and R. Melhem. “An Array Data Flow Analysis based Communication Optimizer„. In Proceedings of the 10th Workshop on Languages and Compilers for Parallel Computing, Minneapolis, MN, Aug. 1997.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Jodi Tims
    • 1
  • Rajiv Gupta
    • 1
  • Mary Lou Soffa
    • 1
  1. 1.Department of Computer ScienceUniversity of PittsburghPittsburgh

Personalised recommendations