Structure-Based Communication Trace Compression

Abstract

Communication traces are increasingly important, both for the performance analysis and optimization of parallel applications and for designing next-generation HPC systems. Meanwhile, the problem size and the execution scale on supercomputers keep growing, producing a prohibitive volume of communication traces. To reduce the size of communication traces, existing dynamic compression methods introduce large compression overhead as the job scale increases. We propose a hybrid static-dynamic method, called Cypress, which leverages information acquired from static analysis to facilitate more effective and efficient dynamic trace compression. Cypress extracts a program communication structure tree at compile time using inter-procedural analysis. This tree naturally contains crucial iterative computing features such as the loop structure, allowing subsequent runtime compression to “fill in,” in a “top-down” manner, event details into the known communication template. Results show that Cypress reduces intra-process and inter-process compression overhead up to 5$\times$ and 9$\times$, respectively, over state-of-the-art dynamic methods while only introducing very low compiling overhead. (Ⓒ 2014 IEEE. Reproduced, with permission, from Jidong Zhai, et al., Cypress: combining static and dynamic analysis for top-down communication trace compression, SC’14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2014.)