Abstract
Many parallel programs are written in a single-program, multiple-data (SPMD) style, in which synchronization is provided using collective operations that all threads execute simultaneously. If these operations are not properly aligned on all threads, deadlock can occur, and many compiler analyses and optimizations that depend on proper alignment fail. In this paper, we discuss the flaws in the Titanium language’s type system for enforcing textual alignment of collectives. We then present a system that uses runtime checks to ensure alignment for two definitions of textual alignment. The system instruments the code to keep track of alignment in each thread and then checks that alignment matches prior to performing a collective operation. We have implemented the system in the Titanium compiler, verifying that it catches alignment errors. We tested its performance on multiple application programs, demonstrating that the checks have no appreciable impact on execution time.
This work was supported in part by the Department of Energy under DE-FC03-01ER25509, FDDE-FC02-07ER25799, and Lawrence Berkeley National Laboratory Contract DE-AC02-05CH11231, by the National Science Foundation under CNS-0325873 and OCI-0749190, by the California State MICRO Program, by Microsoft (Award #024263) and Intel (Award #024894) funding and by matching funding by U.C. Discovery (Award #DIG07-10227), and by gifts from Sun Microsystems. The information presented here does not necessarily reflect the position or the policy of the Government and no official endorsement should be inferred.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Eclipse Parallel Tools Platform, http://www.eclipse.org/ptp/
Aiken, A., Gay, D.: Barrier inference. In: Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (January 1998)
Bailey, D.H., Barszcz, E., Barton, J.T., Browning, D.S., Carter, R.L., Dagum, D., Fatoohi, R.A., Frederickson, P.O., Lasinski, T.A., Schreiber, R.S., Simon, H.D., Venkatakrishnan, V., Weeratunga, S.K.: The NAS Parallel Benchmarks. The International Journal of Supercomputer Applications 5(3), 63–73 (Fall 1991)
Bonachea, D.: GASNet specification, v1.1. Technical Report UCB/CSD-02-1207, University of California, Berkeley (November 2002)
Carlson, W., Draper, J., Culler, D., Yelick, K., Brooks, E., Warren, K.: Introduction to UPC and language specification. Technical Report CCS-TR-99-157, IDA Center for Computing Sciences (1999)
Darte, A., Schreiber, R.: A linear-time algorithm for optimal barrier placement. In: PPoPP 2005: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 26–35. ACM, New York (2005)
Datta, K.: The NAS Parallel Benchmarks in Titanium. Master’s thesis, University of California, Berkeley (December 2005)
Datta, K., Bonachea, D., Yelick, K.: Titanium performance and potential: an NPB experimental study. In: Ayguadé, E., Baumgartner, G., Ramanujam, J., Sadayappan, P. (eds.) LCPC 2005. LNCS, vol. 4339, pp. 200–214. Springer, Heidelberg (2006)
Gay, D.: Barrier Inference. PhD thesis, University of California, Berkeley (May 1998)
Jeremiassen, T., Eggers, S.: Static analysis of barrier synchronization in explicitly parallel programs. In: Proceedings of the IFIP WG10.3 Working Conference on Parallel Architectures and Compilation Techniques (August 1994)
Kamil, A.: Problems with the titanium type system for alignment of collectives (February 2006), http://www.cs.berkeley.edu/~kamil/titanium/doc/single.pdf
Kamil, A., Su., J., Yelick, K.: Making sequential consistency practical in Titanium. In: Supercomputing 2005 (November 2005)
Kamil, A., Yelick, K.: Concurrency analysis for parallel programs with textually aligned barriers. In: Ayguadé, E., Baumgartner, G., Ramanujam, J., Sadayappan, P. (eds.) LCPC 2005. LNCS, vol. 4339, pp. 185–199. Springer, Heidelberg (2006)
Krishnamurthy, A., Yelick, K.: Analyses and optimizations for shared address space programs (1996)
Lin, Y.: Static Nonconcurrency Analysis of OpenMP Programs. In: Mueller, M.S., Chapman, B.M., de Supinski, B.R., Malony, A.D., Voss, M. (eds.) IWOMP 2005 and IWOMP 2006. LNCS, vol. 4315, pp. 36–50. Springer, Heidelberg (2005)
Numwich, R., Reid, J.: Co-Array Fortran for parallel programming. Technical Report RAL-TR-1998-060, Rutherford Appleton Laboratory (1998)
O’Boyle, M., Stohr, E.: Compile time barrier synchronization minimization. IEEE Transactions on Parallel and Distributed Systems 13(6), 529–543 (2002)
Siegel, S.F., Avrunin, G.S.: Modeling wildcard-free MPI programs for verification. In: PPoPP 2005: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 95–106. ACM, New York (2005)
Tseng, C.-W.: Compiler optimizations for eliminating barrier synchronization. SIGPLAN Not. 30(8), 144–155 (1995)
Yelick, K., Semenzato, L., Pike, G., Miyamoto, C., Liblit, B., Krishnamurthy, A., Hilfinger, P., Graham, S., Gay, D., Colella, P., Aiken, A.: Titanium: A high-performance Java dialect. In: Workshop on Java for High-Performance Network Computing, Stanford, California (February 1998)
Zhang, Y., Duesterwald, E.: Barrier matching for programs with textually unaligned barriers. In: PPoPP 2007: Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 194–204. ACM, New York (2007)
Zhang, Y., Duesterwald, E., Gao, G.R.: Concurrency analysis for shared memory programs with textually unaligned barriers. In: Adve, V., Garzarán, M.J., Petersen, P. (eds.) LCPC 2007. LNCS, vol. 5234, pp. 95–109. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kamil, A., Yelick, K. (2010). Enforcing Textual Alignment of Collectives Using Dynamic Checks . In: Gao, G.R., Pollock, L.L., Cavazos, J., Li, X. (eds) Languages and Compilers for Parallel Computing. LCPC 2009. Lecture Notes in Computer Science, vol 5898. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13374-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-13374-9_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13373-2
Online ISBN: 978-3-642-13374-9
eBook Packages: Computer ScienceComputer Science (R0)