A comparative characterization of communication patterns in applications using MPI and shared memory on an IBM SP2

  • Sven Karlsson
  • Mats Brorsson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1362)


In this paper we analyze the characteristics of communication in three different applications, FFT, Barnes and Water, on an IBM SP2. We contrast the communication using two different programming models: message-passing, MPI, and shared memory, represented by a state-of-the-art distributed virtual shared memory package, TreadMarks. We show that while communication time and busy times are comparable for small systems, the communication patterns are fundamentally different leading to poor performance for TreadMarks-based applications when the number of processors increase. This is due to the request/reply technique used in TreadMarks that results in a large fraction of very small messages. However, if the application can be tuned to reduce the impact of small message communication it is possible to achieve acceptable performance at least up to 32 nodes. Our measurements also show that TreadMarks programs tend to cause a more even network load compared to MPI programs.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    C. Amza, A.L. Cox, S. Dwarkadas, P. Keleher, H. Lu, R. Rajamony, W. Yu and W. Zwaenepoel.: Tread Marks: Shared Memory Computing on Networks of Workstations, IEEE Computer, Vol. 29, no. 2, pp. 18–28, February 1996.Google Scholar
  2. 2.
    D. Bailey, T. Harris, W. Saphir, R vd Wijngaart, A. Woo, and M. Yarrow, The NAS Parallel Benchmarks 2.0, Report NAS-95-020, Nasa Ames Research Center, Moffett Field, Ca, 94035, USA. December, 1995.Google Scholar
  3. 3.
    K. Gharachorloo, D. E. Lenoski, J. P. Laudon, P. Gibbons, A. Gupta, and J. L. Hennessy. Memory Consistency and Event Ordering in Scalable Shared-Memory Multiprocessors. In Proceedings of the 17th Annual International Symposium on Computer Architecture, pp. 15–26, May 1990.Google Scholar
  4. 4.
    H. Lu, S. Dwarkadas, A. L. Cox, and W. Zwaenepoel, Quantifying the Performance Differences Between PVM and TreadMarks, Journal of Parallel and Distributed Computing, Vol.43, No. 2, pp. 65–78, June 1997.CrossRefGoogle Scholar
  5. 5.
    Message Passing Interface Forum, MPI: A Message-Passing Interface Standard, version 1.1, June 12, 1995.Google Scholar
  6. 6.
    J. Miguel, A. Arruabarrena, R. Beivide and J. A. Gregorio, Assessing the Performance of the New IBM SPS Communication Subsystem, IEEE Parallel & Distributed Technology, Winter 1996, pp. 12–22.CrossRefGoogle Scholar
  7. 7.
    J. P. Singh, W.-D. Weber, and A. Gupta. SPLASH: Stanford parallel applications for shared-memory. Computer Architecture News, 20(1):5–44, March 1992.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Sven Karlsson
    • 1
  • Mats Brorsson
    • 1
  1. 1.Department of Information TechnologyLund UniversityLundSweden

Personalised recommendations