Measuring the Scalability of Heterogeneous Parallel Systems
A parallel algorithm cannot be evaluated apart from the architecture it is implemented on. So, we define a parallel system as the combination of a parallel algorithm and a parallel architecture. The paper is devoted to the extension of well-known isoefficiency scalability metrics to heterogeneous parallel systems. Based on this extension the scalability of SUMMA (Scalable Universal Matrix Multiplication Algorithm) on parallel architecture with homogeneous communication system supporting simultaneous point-to-point communications is evaluated. Two strategies of data distribution are considered: (i) homogeneous – data are distributed between processors evenly; (ii) data are distributed between processors according to their performance. It is shown that under some assumption both strategies ensure the same scalability of heterogeneous parallel system. This theoretical result is corroborated with experiment.
KeywordsProblem Size Parallel Algorithm Parallel System Parallel Architecture Primary Memory
Unable to display preview. Download preview PDF.
- 4.Quinn, M.: Parallel Programming in C with MPI and OpenMP. McGraw-Hill, New York (2004)Google Scholar
- 5.Pastor, L., Bosque, J.L.: An efficiency and scalability model for heterogeneous clusters. In: Proceedings of Cluster 2001, Newport Beach, CA, USA, October 8-11, 2001, pp. 427–434. IEEE Computer Society, Los Alamitos (2001)Google Scholar
- 6.Kalinov, A.: Scalability Analysis of Matrix-Matrix Multiplication on Heterogeneous Clusters. In: Proceedings of 3rd ISPDC/HeteroPar 2004, Cork, Ireland, July 05 - 07, 2004, pp. 303–309. IEEE CS Press, Los Alamitos (2004)Google Scholar
- 9.Dovolnov, E., Kalinov, A., Klimov, S.: Natural Block Data Decomposition for Heterogeneous Clusters. In: Proceedings of 17th International Parallel and Distributed Processing Symposium, IEEE CS, Nice (April 2003) CD-ROMGoogle Scholar