Performance Analysis of the NWChem TCE for Different Communication Patterns
- Cite this paper as:
- Ghosh P., Hammond J.R., Ghosh S., Chapman B. (2014) Performance Analysis of the NWChem TCE for Different Communication Patterns. In: Jarvis S., Wright S., Hammond S. (eds) High Performance Computing Systems. Performance Modeling, Benchmarking and Simulation. PMBS 2013. Lecture Notes in Computer Science, vol 8551. Springer, Cham
One-sided communication is a model that separates communication from synchronization, and has been in practice for over two decades in libraries such as SHMEM and Global Arrays (GA). GA is used in a number of application codes, especially NWChem, and provides a superset of SHMEM functionality that includes remote accumulate, among other features. Remote accumulate is an active-message operation that applies \(y+=a*x\) at the target rather than just \(y=x\) (as in Put) which gives the programmer additional choices with respect to algorithm design. In this paper, we discuss and evaluate communication scenarios for dense block-tensor contractions, one of the mainstays of the NWChem computation chemistry package. We show that apart from the classical approach involving dynamic scheduling of data blocks for load balancing, reordering one-sided Get and Accumulate calls affects the performance of tensor contractions on leadership-class machines substantially. In order to understand why this reordering affects the performance, we develop a proxy application for the NWChem Tensor Contraction Engine (TCE) module. We utilize this proxy application to compare different implementations with a focus on communication.
KeywordsNWChem One-sided communication Global Arrays MPI-3 Tensor contractions
Unable to display preview. Download preview PDF.