Synchronization issues in data-parallel languages

  • Sundeep Prakash
  • Maneesh Dhagat
  • Rajive Bagrodia
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 768)


Data-parallel programming has established itself as the preferred way of programming a large class of scientific applications. In this paper, we address the issue of reducing synchronization costs when implementing a data-parallel language on an asynchronous architecture. The synchronization issue is addressed from two perspectives: first, we describe language constructs that allow the programmer to specify that different parts of a data-parallel program be synchronized at different levels of granularity. Secondly, we show how existing tools and algorithms for data dependency analysis can be used by the compiler to both reduce the number of barriers and to replace global barriers by cheaper clustered synchronizations. Although the techniques presented in the paper are general purpose, we describe them in the context of a data-parallel language called UC developed at UCLA. Reducing the number of barriers improves program execution time by reducing synchronization time and also processor stall times.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [ABCD93]
    V. Austel, R. Bagrodia, M. Chandy, and M. Dhagat. Reductions + Relations = Data-Parallelism Technical report, University of California, Los Angeles, CA 90024, April 1993.Google Scholar
  2. [AK87]
    R. Allen and K. Kennedy. Automatic Translation of FORTRAN Program to Vector Form. ACM Transactions on Programming Languages and Systems, 9(4):491–543, October 1987.CrossRefGoogle Scholar
  3. [BA92]
    R. Bagrodia and V. Austel. UC User Manual. Computer Science Department, University of California at Los Angeles, 1992.Google Scholar
  4. [BENP93]
    U. Banerjee, R. Eigenmann, A. Nicolau, and D.A. Padua. Automatic Program Parallelization. Proceedings of the IEEE, 81(2):1–33, February 1993.CrossRefGoogle Scholar
  5. [For92]
    High Performance Fortran Forum. High Performance Fortran Language Specification. DRAFT, November 1992.Google Scholar
  6. [FOW87]
    J. Ferrante, K. Ottenstein, and J.D. Warren. The program dependence graph and its use in optimization. ACM TOPLAS, 9(3), July 1987.Google Scholar
  7. [Gav72]
    F. Gavril. Algorithms for minimum coloring, maximum clique, minimum covering by cliques, and maximum independent set of a chordal graph. SIAM Journal on Computing, 1(2):180–187, 1972.CrossRefGoogle Scholar
  8. [Gup89]
    Rajiv Gupta. The fuzzy barrier: A mechanism for high-speed synchronization of processors. In Proceedings of the 3rd International Conference on Architectural Support for Programming Languages and Operating Systems, pages 54–64, April 1989.Google Scholar
  9. [HQL+91]
    P. Hatcher, M. Quinn, A. Lapadula, B. Seevers, R. Anderson, and R. Jones. Data-parallel programming on MIMD computers. IEEE Trans. on Parallel and Distributed Systems, July 1991.Google Scholar
  10. [MA87]
    S. K. Midkiff and Padua D. A. Compiler Algorithms for Synchronization. IEEE Transactions on Computers, C-36(12):1485–1495, December 1987.Google Scholar
  11. [MR90]
    P. Mehrotra and J. Van Rosendale. Programming distributed memory architectures using Kali. Report 90-69, Institute for Computer Application in Science and Engineering, Hampton, VA, 1990.Google Scholar
  12. [PW86]
    D.A. Padua and M.J. Wolfe. Advanced Compiler Optimizations for Supercomputers. Communications of the ACM, 29(12):1184–1201, December 1986.CrossRefGoogle Scholar
  13. [QHS91]
    M. Quinn, P. Hatcher, and B. Seevers. Implementing a Data Parallel Language on a Tightly Coupled Multiprocessor. In A. Nicolau, D. Gelernter, Gross T., and Padua D., editors, Advances in Languages and Compilers for Parallel Processing, chapter 20, pages 385–401. MIT Press, Cambridge, Massachusetts, 1991.Google Scholar
  14. [RS87]
    J.R. Rose and G.L. Steele. C*: An Extended C Language for Data Parallel Programming. PL-87.5, Thinking Machines Corporation, March 1987.Google Scholar
  15. [RSW91]
    M. Rosing, R. B. Schnabel, and R. B. Weaver. The DINO Parallel Programming Language. Journal of Parallel and Distributed Computing, 13:30–42, 1991.CrossRefGoogle Scholar
  16. [SSS88]
    C. L. Seitz, J. Seizovic, and W. K. Su. The C Programmer's Abbreviated Guide to Multicomputer Programming. Technical Report Caltech-CS-TR-88-1, Caltech, January 1988.Google Scholar
  17. [Wol89]
    Michael Wolfe. Optimizing Supercompilers for Supercomputers. MIT Press, 1989.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1994

Authors and Affiliations

  • Sundeep Prakash
    • 1
  • Maneesh Dhagat
    • 1
  • Rajive Bagrodia
    • 1
  1. 1.Computer Science DepartmentUniversity of CaliforniaLos Angeles

Personalised recommendations