Communication Models for Resource Constrained Hierarchical Ethernet Networks

Abstract

Communication time prediction is critical for parallel application performance tuning, especially for the rapidly growing field of data-intensive applications. However, making such predictions accurately is non-trivial when contention exists on different components in hierarchical networks. In this paper, we derive an ‘asymmetric network property’ on TCP layer for concurrent bidirectional communications in a commercial off-the-shelf (COTS) cluster, and develop a communication model as the first effort to characterize the communication times on hierarchical Ethernet networks with contentions on both NIC and backbone cable levels. In particular, we show that if the asymmetric network property was excluded from the model, the communication time predictions will be significantly less accurate than those made by using the asymmetric network property. Furthermore, our observation of the performance degradation caused by the asymmetric network property suggests that some part of the software stack below TCP layer in COTS clusters needs targeted tuning, which has not yet attracted any attention in literature.