Performance analysis and prediction for distributed homogeneous clusters
- 183 Downloads
We present a new performance model based on the roofline concept for the analysis and performance prediction of distributed computing clusters. The background for our performance modeling is the 28 km InfiniBand interconnection between two bwGRiD clusters each consisting of 140 compute nodes in day-to-day production use. The model is used to analyze the MPI performance of intra-cluster communication compared to inter-cluster communication. We compare the new modeling results to our earlier stochastic model (Richling et al. in Proc. of 3PGCIC-2010. IEEE, New York 2010) where we could give an estimate on the bandwidth requirements for doubling the performance of an application (LinPack as the simplest example). We will derive some bounds for the size of regions in a cluster and the scaling of the maximal speed-up for the region-region-interconnected network.
KeywordsPerformance model Performance prediction Inter-cluster communication Roofline model
We thank our colleagues Rolf Bogus, Hermann Lauer and Steffen Hau as well as the bwGRiD team for the help in the construction and operation of the interesting hardware and the optimization of the connection, which is the basis for this paper. One of the authors, H.G. Kruse, thanks LBNL/Berkeley for the hospitality during a research visit. The inspiring and exciting atmosphere favored the becoming of this work. We also thank the referees for the insightful suggestions to improve the paper.
bwGRiD is a Member of the German D-Grid initiative and is funded by the Ministry of Education and Research and the Ministry for Science, Research and Arts Baden-Württemberg.
- 1.bwGRiD (2007–2010) Member of the German D-Grid initiative, funded by the Ministry of Education and Research and the Ministry for Science, Research and Arts Baden-Württemberg, Universities of Baden-Württemberg, . http://www.bw-grid.de/. Accessed May 2010
- 2.Kredel H, Kruse H-G, Richling S (2010) Zur Leistung von verteilten, homogenen Clustern. Pik 2:166–171 Google Scholar
- 3.Richling S, Hau S, Kredel H, Kruse H-G (2010) Operating two InfiniBand Grid clusters over 28 km distance. In: Proc of 3PGCIC-2010. IEEE, New York Google Scholar
- 5.Richling S, Hau S, Kredel H, Kruse H-G (2011) A long-distance InfiniBand interconnection between two clusters in production use. In: Proc supercomputing, November 12–18, 2011. IEEE, New York Google Scholar
- 6.Merz M, Krietemeyer M (eds) (2006) IPACS integrated performance analysis of computer systems—benchmarks for distributed computer systems. Logos Verlag, Berlin Google Scholar
- 7.Obsidian: high performance network. http://www.obsidianresearch.com/. Accessed May 2010
- 8.LinPack und HPL: Linear Algebra Package and High Performance LinPack. http://www.netlib.org/benchmark/hpl/. Accessed Jan 2012
- 9.Schlegel U, Grobe K, Southwell D 100 Gbit/s DWDM InfiniBand transport over up to 40 km. http:tnc2009.terena.org/core/getfile7b9a.pdf?file_id=10. Accessed May 2010
- 12.Carter S, Minich M, Rao N (2007) Experimental evaluation of InfiniBand transport over local- and wide-area networks. In: Proc 2007 spring simulation multiconference (SpringSim’07), pp 419–426 Google Scholar
- 13.Kredel H, Kruse H-G, Ott I (2011) Lastverhalten und Systemkonfiguration von Web-Applikationsservern. Praxis d. Informationsverarbeitung und Kommunikation (PIK), vol 3. De Gruyter Saur, München, pp 215–223 Google Scholar
- 14.Kredel H, Kruse H-G, Ott I (2011) Performance analysis and performance modeling of web-applications. In: Proc 3PGCIC-2011. IEEE, New York, pp 115–122 Google Scholar
- 17.Popa A (2012) What is the speed of light in a fiber optic cable? http://www.madsci.org/posts/archives/2001-02/983369337.Ph.r.html. Accessed Jan 2012