An Embedded DSL for High Performance Declarative Communication with Correctness Guarantees in C++
High performance programming using explicit communication calls needs considerable programming expertise to optimize. Tuning for performance often involves using asynchronous calls, running the risk of introducing bugs and making the program harder to debug. Techniques to prove desirable program properties, such as deadlock freedom, invariably incur significant performance overheads.
We have developed a domain-specific language, embedded in C++, called Kanor that enables programmers to specify the communication declaratively in the Bulk Synchronous Parallel (BSP) style. Deadlock freedom is guaranteed for well-formed Kanor programs. We start with operational semantics for a subset of Kanor and prove deadlock freedom and determinism properties based on those semantics. We then show how the declarative nature of Kanor allows us to detect and optimize communication patterns.
- 3.Cottam, J.A., Holk, E., Byrd, W.E., Chauhan, A., Lumsdaine, A.: High-level coordination specification: operational semantics for kanor. In: Workshop on Leveraging Abstractions and Semantics in High-Performance Computing (LASH-C; Workshop at PPoPP 2013), February 2013Google Scholar
- 5.Danalis, A., Pollock, L., Swany, M., Cavazos, J.: Mpi-aware compiler optimizations for improving communication-computation overlap. In Proceedings of the 23rd International Conference on Supercomputing, ICS 2009, pp. 316–325. ACM, New York (2009)Google Scholar
- 6.Gava, F., Fortin, J.: Formal semantics of a subset of the paderborn’s BSPlib. In: Proceedings of the Ninth International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2008, pp. 269–276. IEEE Computer Society, Washington, DC (2008)Google Scholar
- 8.Hoefler, T., Schneider. T.: Runtime detection and optimization of collective communication patterns. In: Proceedings of the 21st International Conference on Parallel Architectures and Compilation Techniques (PACT), pp. 263–272. ACM (2012)Google Scholar
- 9.Jiao, F., Mahajan, N., Willcocok, J., Chauhan, A., Lumsdaine, A.: Partial globalization of partitioned address spaces for zero copy communication with shared memory. In: Proc. of the 18th International Conference on High Performance Computing (HiPC) (2011). doi:10.1109/HiPC.2011.6152733
- 11.Valiant, L.G.: Bulk-synchronous parallel computers. In: Reeve, M. (ed.) Parallel Processing and Artificial Intelligence, pp. 15–22. John Wiley & Sons (1989)Google Scholar