Load Imbalance in Parallel Programs

  • Maria Calzarossa
  • Luisa Massari
  • Daniele Tessera
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2763)


Parallel programs experience performance inefficiencies as a result of dependencies, resource contentions, uneven work distributions and loss of synchronizations among processors. The analysis of these inefficiencies is very important for tuning and performance debugging studies. In this paper we address the identification and localization of performance inefficiencies from a methodological viewpoint. We follow a top down approach. We first analyze the performance properties of the programs at a coarse grain. We then study the behavior of the processors and their load imbalance. The methodology is illustrated on a study of a message passing computational fluid dynamic program.


Code Region Performance Property Parallel Program Wall Clock Time Load Imbalance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Calzarossa, M., Massari, L., Merlo, A., Pantano, M., Tessera, D.: Medea: A Tool for Workload Characterization of Parallel Systems. IEEE Parallel and Distributed Technology 3(4), 72–80 (1995)CrossRefGoogle Scholar
  2. 2.
    De Rose, L., Zhang, Y., Reed, D.A.: SvPablo: A Multi-Language Performance Analysis System. In: Puigjaner, R., Savino, N.N., Serra, B. (eds.) TOOLS 1998. LNCS, vol. 1469, pp. 352–355. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  3. 3.
    Ferschweiler, K., Harrah, S., Keon, D., Calzarossa, M., Tessera, D., Pancake, C.: The Tracefile Testbed – A Community Repository for Identifying and Retrieving HPC Performance Data. In: Proc. 2002 International Conference on Parallel Processing, pp. 177–184. IEEE Press, Los Alamitos (2002)CrossRefGoogle Scholar
  4. 4.
    Hartigan, J.A.: Clustering Algorithms. Wiley, Chichester (1975)zbMATHGoogle Scholar
  5. 5.
    Heath, M.T., Etheridge, J.A.: Visualizing the Performance of Parallel Programs. IEEE Software 8, 29–39 (1991)CrossRefGoogle Scholar
  6. 6.
    Helm, B., Malony, A., Fickas, S.: Capturing and Automating Performance Diagnosis: the Poirot Approach. In: Proceedings of the 1995 International Parallel Processing Symposium, pp. 606–613 (1995)Google Scholar
  7. 7.
    Karavanic, K.L., Miller, B.P.: Improving Online Performance Diagnosis by the Use of Historical Performance Data. In: Proc. SC 1999 (1999)Google Scholar
  8. 8.
    Marshall, A.W., Olkin, I.: Inequalities: Theory of Majorization and Its Applications. Academic Press, London (1979)zbMATHGoogle Scholar
  9. 9.
    Miller, B.P., Callaghan, M.D., Cargille, J.M., Hollingsworth, J.K.H., Irvin, R.B., Karavanic, K.L., Kunchithapadam, K., Newhall, T.: The Paradyn Parallel Measurement Performance Tool. IEEE Computer 28(11), 37–46 (1995)CrossRefGoogle Scholar
  10. 10.
    Roth, P.C., Miller, B.P.: Deep Start: A Hybrid Strategy for Automated Performance Problem Searches. In: Monien, B., Feldmann, R.L. (eds.) Euro-Par 2002. LNCS, vol. 2400, pp. 86–96. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Simmons, M.L., Hayes, A.H., Brown, J.S., Reed, D.A. (eds.): Debugging and Performance Tuning for Parallel Computing Systems. IEEE Computer Society, Los Alamitos (1996)Google Scholar
  12. 12.
    Williams, W., Hoel, T., Pase, D.: The MPP Apprentice Performance Tool: Delivering the Performance of the Cray T3D. In: Decker, K.M. (ed.) Programming Environments for Massively Parallel Distributed Systems, pp. 333–345. Birkhauser, Basel (1994)CrossRefGoogle Scholar
  13. 13.
    Yan, J.C., Sarukkai, S.R.: Analyzing Parallel Program Performance Using Normalized Performance Indices and Trace Transformation Techniques. Parallel Computing 22(9), 1215–1237 (1996)CrossRefzbMATHGoogle Scholar
  14. 14.
    Zaki, O., Lusk, E., Gropp, W., Swider, D.: Toward Scalable Performance Visualization with Jumpshot. The International Journal of High Performance Computing Applications 13(2), 277–288 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Maria Calzarossa
    • 1
  • Luisa Massari
    • 1
  • Daniele Tessera
    • 1
  1. 1.Dipartimento di Informatica e SistemisticaUniversità di PaviaPaviaItaly

Personalised recommendations