Comparing Performance Heatmaps

  • David Krakov
  • Dror G. FeitelsonEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8429)


The performance of parallel job schedulers is often expressed as an average metric value (e.g. response time) for a given average load. An alternative is to acknowledge the wide variability that exists in real systems, and use a heatmap that portrays the distribution of jobs across the performance \(\times \) load space. Such heatmaps expose a wealth of details regarding the conditions that occurred in production use or during a simulation. However, heatmaps are a visual tool, lending itself to high-resolution analysis of a single system but not conducive for a direct comparison between different schedulers or environments. We propose a number of techniques that allow to compare heatmaps. The first two treat the heatmaps as images, and focus on the differences between them. Two other techniques are based on tracking how specific jobs fare under the compared scenarios, and drawing underlying trends. This enables a detailed analysis of how different schedulers affect the workload, and what leads to the observed average results.


Wait Time Ratio Difference High Resolution Analysis Picket Fence Real World Trace 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



Many thanks to all those who have made their workload data available through the Parallel Workloads Archive.


  1. 1.
    Anscombe, F.J.: Graphs in Statistical Analysis. Am. Stat. 27(1), 17–21 (1973)Google Scholar
  2. 2.
    Feitelson, D.G.: Looking at data. In: 22nd International Parallel & Distributed Processing Symposium (IPDPS), April 2008Google Scholar
  3. 3.
    Chapin, S.J., Cirne, W., Feitelson, D.G., Jones, J.P., Leutenegger, S.T., Schwiegelshohn, U., Smith, W., Talby, D.: Benchmarks and standards for the evaluation of parallel job schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1999, IPPS-WS 1999, and SPDP-WS 1999. LNCS, vol. 1659, pp. 67–90. Springer, Heidelberg (1999) CrossRefGoogle Scholar
  4. 4.
    Crovella, M.E.: Performance evaluation with heavy tailed distributions. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 1–9. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  5. 5.
    Downey, A.B., Feitelson, D.G.: The elusive goal of workload characterization. Perform. Eval. Rev. 26(4), 14–29 (1999)CrossRefGoogle Scholar
  6. 6.
    Etsion, Y., Tsafrir, D., Feitelson, D.G.: Process prioritization using output production: scheduling for multimedia. ACM Trans. Multimed. Comput. Commun. Appl. 2(4), 318–342 (2006)CrossRefGoogle Scholar
  7. 7.
    Frachtenberg, E., Feitelson, D.G., Petrini, F., Fernandez, J.: Adaptive parallel job scheduling with flexible coscheduling. IEEE Trans. Parallel Distrib. Syst. 16(11), 1066–1077 (2005)CrossRefGoogle Scholar
  8. 8.
    Krakov, D., Feitelson, D.G.: High-resolution analysis of parallel job workloads. In: Cirne, W., Desai, N., Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2012. LNCS, vol. 7698, pp. 178–195. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  9. 9.
    Lifka, D.: The ANL/IBM SP scheduling system. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1995. LNCS, vol. 949, pp. 295–303. Springer, Heidelberg (1995)CrossRefGoogle Scholar
  10. 10.
    Feitelson, D.G., Rudoplh, L., Schwiegelshohn, U., Sevcik, K.C., Wong, P.: Theory and practice in parallel job scheduling. JSSPP 1997. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Rudolph, L., Smith, P.H.: Valuation of ultra-scale computing systems. In: Feitelson, D.G., Rudolph, L. (eds.) IPDPS-WS 2000 and JSSPP 2000. LNCS, vol. 1911, pp. 39–55. Springer, Heidelberg (2000) CrossRefGoogle Scholar
  13. 13.
    Performance Heatmap Utilities.

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.School of Computer Science and EngineeringThe Hebrew University of JerusalemJerusalemIsrael

Personalised recommendations